Re: Sharing libc in ELF modules.

[Gregory Nutt]

The way that has been done in the past was too add the libc functions 
to the base FLASH symbol table.  That symbol table draws the libc 
functions into the base FLASH image*.  Then there is one copy of the 
libc functions in base FLASH and no libc functions in the ELF 
module.  The ELF module is linked to the libc functions just as they 
are linked to the OS inteface functions.


(And I forgot ... don't link the ELF module with libc.a so that all of 
the libc symbols are undefined and will get the values from the base 
FLASH symbol table when the ELF module is dynamically loaded.

Re: Google Summer of Code

[Roberto Bucher]

Hi

for me it's ok to do a meeting today at 11:00 UTC. Do I have to organize 
a zoom meeting?


BR

Roberto

On 5/18/21 9:22 AM, Pavel Pisa wrote:

Hello Roberto and others,

I am happy that the project has been selected and hope that
it helps to advance usesfull toll and OS/HW combination.

My availability, today 11:00 to 12:00 UTC, then after 18:00 UTC.
Tomorrow, Wednesday till 11:00 UTC and after 18:00 UTC.
Thusday is ideal, I can schedule activities at my will.
I want to have leave on this Friday and weekend, we plan
boat trip. Next week Tueday and even better Thursday
and evenings after 18:00 UTC other days are possible.

Best wishes,

Pavel

PS: we should probably define some place where to collect
 information to coordinate the work. May it be corresponding
 NuttX Apache issue or pysimCoder Wiki, NuttX Wiki
 Michal Lenc's GitHub wiki

On Tuesday 18 of May 2021 07:34:42 Roberto Bucher wrote:

Buongiorno Michal

I'd like to congratulate with you about the acceptance of the NuttX
project "NuttX Support for RapidControl Applications Development with
pysimCoder" a the GSOC!

I'm very excited to share with you my experience during this project,
and I hope that your contribution can help in develop and sharing
pysimCoder with the community.

I think that we can organize ASAP a video meeting, perhaps with Prof.
Pisa and Brennan Ashton.

Best regards

Prof. Roberto Bucher

Re: Google Summer of Code

[lencmich]

Hello all,

firstly I would like to thank you all for your support during the  
proposal period. I am looking forward to cooperating on pysimCoder  
integration with NuttX.


I forked pysimCoder GIT few days ago and started with implementation  
of SocketCAN support. I can successfully send and receive CAN  
messages, thought RT receive is yet to be done. The GIT branch can be  
seen at <.


Regarding the meeting, I have a consultation for semestral project  
starting at 11 UTC. I have just a one question about the project, so I  
expect it would take just a few minutes, would it be ok to move our  
meeting to 11:15 UTC?


Regarding some place for updates about the project, I think NuttX  
GitHub Issue would be the most appropriate as it should have the most  
amount of readers.


Best regards,
Michal Lenc


Cituji Roberto Bucher :


Hi

for me it's ok to do a meeting today at 11:00 UTC. Do I have to  
organize a zoom meeting?


BR

Roberto

On 5/18/21 9:22 AM, Pavel Pisa wrote:

Hello Roberto and others,

I am happy that the project has been selected and hope that
it helps to advance usesfull toll and OS/HW combination.

My availability, today 11:00 to 12:00 UTC, then after 18:00 UTC.
Tomorrow, Wednesday till 11:00 UTC and after 18:00 UTC.
Thusday is ideal, I can schedule activities at my will.
I want to have leave on this Friday and weekend, we plan
boat trip. Next week Tueday and even better Thursday
and evenings after 18:00 UTC other days are possible.

Best wishes,

Pavel

PS: we should probably define some place where to collect
information to coordinate the work. May it be corresponding
NuttX Apache issue or pysimCoder Wiki, NuttX Wiki
Michal Lenc's GitHub wiki

On Tuesday 18 of May 2021 07:34:42 Roberto Bucher wrote:

Buongiorno Michal

I'd like to congratulate with you about the acceptance of the NuttX
project "NuttX Support for RapidControl Applications Development with
pysimCoder" a the GSOC!

I'm very excited to share with you my experience during this project,
and I hope that your contribution can help in develop and sharing
pysimCoder with the community.

I think that we can organize ASAP a video meeting, perhaps with Prof.
Pisa and Brennan Ashton.

Best regards

Prof. Roberto Bucher

Re: Google Summer of Code

[Brennan Ashton]

I am in PT so 11:00 UTC is 4AM (and tonight is already a very late night
for me, so if we could possibly do it a little later that would be ideal if
possible.

--Brennan

On Tue, May 18, 2021, 12:59 AM  wrote:

>
> Hello all,
>
> firstly I would like to thank you all for your support during the
> proposal period. I am looking forward to cooperating on pysimCoder
> integration with NuttX.
>
> I forked pysimCoder GIT few days ago and started with implementation
> of SocketCAN support. I can successfully send and receive CAN
> messages, thought RT receive is yet to be done. The GIT branch can be
> seen at <.
>
> Regarding the meeting, I have a consultation for semestral project
> starting at 11 UTC. I have just a one question about the project, so I
> expect it would take just a few minutes, would it be ok to move our
> meeting to 11:15 UTC?
>
> Regarding some place for updates about the project, I think NuttX
> GitHub Issue would be the most appropriate as it should have the most
> amount of readers.
>
> Best regards,
> Michal Lenc
>
>
> Cituji Roberto Bucher :
>
> > Hi
> >
> > for me it's ok to do a meeting today at 11:00 UTC. Do I have to
> > organize a zoom meeting?
> >
> > BR
> >
> > Roberto
> >
> > On 5/18/21 9:22 AM, Pavel Pisa wrote:
> >> Hello Roberto and others,
> >>
> >> I am happy that the project has been selected and hope that
> >> it helps to advance usesfull toll and OS/HW combination.
> >>
> >> My availability, today 11:00 to 12:00 UTC, then after 18:00 UTC.
> >> Tomorrow, Wednesday till 11:00 UTC and after 18:00 UTC.
> >> Thusday is ideal, I can schedule activities at my will.
> >> I want to have leave on this Friday and weekend, we plan
> >> boat trip. Next week Tueday and even better Thursday
> >> and evenings after 18:00 UTC other days are possible.
> >>
> >> Best wishes,
> >>
> >> Pavel
> >>
> >> PS: we should probably define some place where to collect
> >> information to coordinate the work. May it be corresponding
> >> NuttX Apache issue or pysimCoder Wiki, NuttX Wiki
> >> Michal Lenc's GitHub wiki
> >>
> >> On Tuesday 18 of May 2021 07:34:42 Roberto Bucher wrote:
> >>> Buongiorno Michal
> >>>
> >>> I'd like to congratulate with you about the acceptance of the NuttX
> >>> project "NuttX Support for RapidControl Applications Development with
> >>> pysimCoder" a the GSOC!
> >>>
> >>> I'm very excited to share with you my experience during this project,
> >>> and I hope that your contribution can help in develop and sharing
> >>> pysimCoder with the community.
> >>>
> >>> I think that we can organize ASAP a video meeting, perhaps with Prof.
> >>> Pisa and Brennan Ashton.
> >>>
> >>> Best regards
> >>>
> >>> Prof. Roberto Bucher
>
>
>
>

NuttX (Online) Workshop 2021 (call for event team members)

[Alin.Jerpelea]

Hi all,

Yesterday me and Alan were talking about the plans for the NuttX 2021 workshop 
and we were thinking that we should see if there are other people interested to 
be part of the event team.
If you are interested to be part of the event team please replay to this mail 
and we will invite you to the online meeting.
Regards
Alin

Re: Google Summer of Code

[Roberto Bucher]

I have a meeting till for 1:45PM to 3:15PM UTC in the afternoon, so for 
me is possible to do something starting at 3:30PM. Otherwise it is 
possible for me to do an additional meeting with Brennan and Michal 
tomorrow later in the afternoon, and something today with Pavel and 
Michal at 11:15


BR

Roberto

On 5/18/21 10:05 AM, Brennan Ashton wrote:
I am in PT so 11:00 UTC is 4AM (and tonight is already a very late 
night for me, so if we could possibly do it a little later that would 
be ideal if possible.


--Brennan

On Tue, May 18, 2021, 12:59 AM > wrote:



Hello all,

firstly I would like to thank you all for your support during the
proposal period. I am looking forward to cooperating on pysimCoder
integration with NuttX.

I forked pysimCoder GIT few days ago and started with implementation
of SocketCAN support. I can successfully send and receive CAN
messages, thought RT receive is yet to be done. The GIT branch can be
seen at <>.

Regarding the meeting, I have a consultation for semestral project
starting at 11 UTC. I have just a one question about the project,
so I
expect it would take just a few minutes, would it be ok to move our
meeting to 11:15 UTC?

Regarding some place for updates about the project, I think NuttX
GitHub Issue would be the most appropriate as it should have the most
amount of readers.

Best regards,
Michal Lenc


Cituji Roberto Bucher mailto:roberto.buc...@supsi.ch>>:

> Hi
>
> for me it's ok to do a meeting today at 11:00 UTC. Do I have to
> organize a zoom meeting?
>
> BR
>
> Roberto
>
> On 5/18/21 9:22 AM, Pavel Pisa wrote:
>> Hello Roberto and others,
>>
>> I am happy that the project has been selected and hope that
>> it helps to advance usesfull toll and OS/HW combination.
>>
>> My availability, today 11:00 to 12:00 UTC, then after 18:00 UTC.
>> Tomorrow, Wednesday till 11:00 UTC and after 18:00 UTC.
>> Thusday is ideal, I can schedule activities at my will.
>> I want to have leave on this Friday and weekend, we plan
>> boat trip. Next week Tueday and even better Thursday
>> and evenings after 18:00 UTC other days are possible.
>>
>> Best wishes,
>>
>> Pavel
>>
>> PS: we should probably define some place where to collect
>>     information to coordinate the work. May it be corresponding
>>     NuttX Apache issue or pysimCoder Wiki, NuttX Wiki
>>     Michal Lenc's GitHub wiki
>>
>> On Tuesday 18 of May 2021 07:34:42 Roberto Bucher wrote:
>>> Buongiorno Michal
>>>
>>> I'd like to congratulate with you about the acceptance of the
NuttX
>>> project "NuttX Support for RapidControl Applications
Development with
>>> pysimCoder" a the GSOC!
>>>
>>> I'm very excited to share with you my experience during this
project,
>>> and I hope that your contribution can help in develop and sharing
>>> pysimCoder with the community.
>>>
>>> I think that we can organize ASAP a video meeting, perhaps
with Prof.
>>> Pisa and Brennan Ashton.
>>>
>>> Best regards
>>>
>>> Prof. Roberto Bucher

Roberto Bucher's Zoom Meeting

[Roberto Bucher]

Roberto Bucher is inviting you to a scheduled Zoom meeting.

Topic: Roberto Bucher's Zoom Meeting
Time: May 18, 2021 01:00 PM Zurich

Join Zoom Meeting
https://supsi.zoom.us/j/61553219775?pwd=TkcwNkJ3YWhUYXFxa2xmQW5Za0hxUT09

Meeting ID: 615 5321 9775
Passcode: 133097

BR

Roberto

Re: Google Summer of Code

[Alan Carvalho de Assis]

Contratulations Michal!

It is really a great news!

Thank you Pavel, Roberto and Brennan to make it happen!

BR,

Alan

On 5/18/21, Roberto Bucher  wrote:
> Buongiorno Michal
>
> I'd like to congratulate with you about the acceptance of the NuttX
> project "NuttX Support for RapidControl Applications Development with
> pysimCoder" a the GSOC!
>
> I'm very excited to share with you my experience during this project,
> and I hope that your contribution can help in develop and sharing
> pysimCoder with the community.
>
> I think that we can organize ASAP a video meeting, perhaps with Prof.
> Pisa and Brennan Ashton.
>
> Best regards
>
> Prof. Roberto Bucher
>
>

Nuttx FAT32 issues - corrupted files, wrongly stated free clusters by statfs

[Reto Gähwiler]

Dear All,

First of all, in case a similar thread pops-up authored by myself, please
ignore.

Recently we discovered some issues with the FAT32 partition on the SD-Card
used in our device running on nuttx. There are actually a bunch of issues
we are strugle to understand related to the filesystem.
The issue discovered recently is, that a statfs call won't return the true
number of free clusters. It rather returns what ever is in the FS INFO
section of the FAT32 partition. Inserting such a "corrupted" card into an
SD-Card reader and mounting it to Windows shows following:

#1 Windows file explorer reports the correct free space.
#2 Comparing the free space in file explorer and looging at FS INFO section
with "Active - Disk Editor" doesn't line up. Windows wouldn't write any
longer the FS INFO section on that card. Even if all the files/dires are
wiped from the card.
#3 Whenever files are added / removed under nuttx the FS INFO section would
be updated by nuttx, but to the wrong number since the base is wrong.
#4 Mounting the card to linux and properly unmount it might actually fix
the issue. But not always!
#5 Quick format the card resolved the issue too. Windows would once again
write the FS INFO section and nuttx would shows the correct free space.
#5 Running a "chkdsk /f" under windows on that broken SD-Card fixes the
issue too. Reporting a broken file in a root folder on that card.

We indeed had issues with corrupted files in the past. Usually in the
logging directory, which is accessed most. How they corrupt we do not know.
The corruption is recognised in the following ways:

#A cryptic and too long file names, we only make use of short file names
(8.3)
#B files whcih became folders
#C sizes which are not possible
#D combinations of the above

Our application supervises the SD-Card with statfs and removes files once
we hit a quota of 0.85. Therefore, the card is not filling up. However, the
most recent recognition of this behvaiour was due to an application bug
which allowed the card to fill up under some circumstances.
Playing around with a quickformated SD-Card, filled up with data and trying
to retrigger the issue revealed that I never get the ENOSPC but only EIO
once the card is full! But never managed to get the statfs issue.
For the logging we use the fstream family (fopen, fwrite, fread, fflush,
fsync) commands. For config files and other things also the low level
read/write are used.

One more thing, among all the logging files we once in a while have
corrupted file content. Typically that is a skipped byte or doubled byte
(we use protobuf and can see that encoding the binary). This seems to
happen during reading but also writing the file.

At the moment we didn't investigate the fat driver of nuttx but rather
tried to analyse the problem. And now we have a bunch of question marks.

#1 Has anyone recongised similar issues?
#2 Does anyone know a tool which can be used in nuttx equivalent to chkdsk
under windows? Or do quick format?
#3 Why is there no ENOSPC but EIO once the card is full?
#4 What triggers the corrupted files? How robust is the nuttx FAT32
considering stackdumps and powerlosses?

For the purpose of investigation, I have an image of the SD-Card when it
was in the state of reporting wrong sizes in nuttx.

Looking forward to your answers and ideas. Appreciate your help,

Reto

RE: NuttX (Online) Workshop 2021 (call for event team members)

[David Sidrane]

Please count me in.

-Original Message-
From: alin.jerpe...@sony.com [mailto:alin.jerpe...@sony.com]
Sent: Tuesday, May 18, 2021 1:54 AM
To: dev@nuttx.apache.org
Subject: NuttX (Online) Workshop 2021 (call for event team members)

Hi all,

Yesterday me and Alan were talking about the plans for the NuttX 2021
workshop and we were thinking that we should see if there are other people
interested to be part of the event team.
If you are interested to be part of the event team please replay to this
mail and we will invite you to the online meeting.
Regards
Alin

Re: [ANNOUNCEMENT] NuttX Participating in Google Summer of Code

[Nathan Hartman]

On Tue, May 18, 2021 at 12:27 AM Brennan Ashton 
wrote:

> Hey everyone,
> We had multiple students this year propose projects for the NuttX project
> under Apache to Google Summer of Code (GSOC), and I'm excited to share one
> of them was accepted.
>
> Michal Lenc will be working on his project titled: NuttX Support for Rapid
> Control Applications Development with pysimCoder
>
> https://summerofcode.withgoogle.com/projects/#4867567685992448
>
> Michal has already been contributing to the project and I am really excited
> to see what comes from this effort.
>
> We have a few formal mentors already but please welcome and help him out,
> as he works with the project over this summer.
>
> --Brennan
>

Welcome Michal and congratulations on getting your project accepted! I'm
very excited for you and the project.

Cheers,
Nathan

Re: NuttX (Online) Workshop 2021 (call for event team members)

[Abdelatif Guettouche]

Do we have any preliminary dates for the event?

On Tue, May 18, 2021 at 3:02 PM David Sidrane  wrote:
>
> Please count me in.
>
> -Original Message-
> From: alin.jerpe...@sony.com [mailto:alin.jerpe...@sony.com]
> Sent: Tuesday, May 18, 2021 1:54 AM
> To: dev@nuttx.apache.org
> Subject: NuttX (Online) Workshop 2021 (call for event team members)
>
> Hi all,
>
> Yesterday me and Alan were talking about the plans for the NuttX 2021
> workshop and we were thinking that we should see if there are other people
> interested to be part of the event team.
> If you are interested to be part of the event team please replay to this
> mail and we will invite you to the online meeting.
> Regards
> Alin

Re: NuttX (Online) Workshop 2021 (call for event team members)

[Alan Carvalho de Assis]

Not yet, the idea is to discuss about it (5W1H)

BR,

Alan

On 5/18/21, Abdelatif Guettouche  wrote:
> Do we have any preliminary dates for the event?
>
> On Tue, May 18, 2021 at 3:02 PM David Sidrane 
> wrote:
>>
>> Please count me in.
>>
>> -Original Message-
>> From: alin.jerpe...@sony.com [mailto:alin.jerpe...@sony.com]
>> Sent: Tuesday, May 18, 2021 1:54 AM
>> To: dev@nuttx.apache.org
>> Subject: NuttX (Online) Workshop 2021 (call for event team members)
>>
>> Hi all,
>>
>> Yesterday me and Alan were talking about the plans for the NuttX 2021
>> workshop and we were thinking that we should see if there are other
>> people
>> interested to be part of the event team.
>> If you are interested to be part of the event team please replay to this
>> mail and we will invite you to the online meeting.
>> Regards
>> Alin
>

RE: NuttX (Online) Workshop 2021 (call for event team members)

[Alin.Jerpelea]

That is one of the things to be decided during the meeting 

Regards
Alin

-Original Message-
From: Abdelatif Guettouche  
Sent: den 18 maj 2021 17:06
To: dev@nuttx.apache.org
Subject: Re: NuttX (Online) Workshop 2021 (call for event team members)

Do we have any preliminary dates for the event?

On Tue, May 18, 2021 at 3:02 PM David Sidrane  wrote:
>
> Please count me in.
>
> -Original Message-
> From: alin.jerpe...@sony.com [mailto:alin.jerpe...@sony.com]
> Sent: Tuesday, May 18, 2021 1:54 AM
> To: dev@nuttx.apache.org
> Subject: NuttX (Online) Workshop 2021 (call for event team members)
>
> Hi all,
>
> Yesterday me and Alan were talking about the plans for the NuttX 2021 
> workshop and we were thinking that we should see if there are other 
> people interested to be part of the event team.
> If you are interested to be part of the event team please replay to 
> this mail and we will invite you to the online meeting.
> Regards
> Alin

Re: NuttX (Online) Workshop 2021 (call for event team members)

[Flavio Castro Alves Filho]

Hello,

I am also interested in being part of the event team.

Best regards,

Flavio

Em ter., 18 de mai. de 2021 às 12:10,  escreveu:
>
> That is one of the things to be decided during the meeting
>
> Regards
> Alin
>
> -Original Message-
> From: Abdelatif Guettouche 
> Sent: den 18 maj 2021 17:06
> To: dev@nuttx.apache.org
> Subject: Re: NuttX (Online) Workshop 2021 (call for event team members)
>
> Do we have any preliminary dates for the event?
>
> On Tue, May 18, 2021 at 3:02 PM David Sidrane  wrote:
> >
> > Please count me in.
> >
> > -Original Message-
> > From: alin.jerpe...@sony.com [mailto:alin.jerpe...@sony.com]
> > Sent: Tuesday, May 18, 2021 1:54 AM
> > To: dev@nuttx.apache.org
> > Subject: NuttX (Online) Workshop 2021 (call for event team members)
> >
> > Hi all,
> >
> > Yesterday me and Alan were talking about the plans for the NuttX 2021
> > workshop and we were thinking that we should see if there are other
> > people interested to be part of the event team.
> > If you are interested to be part of the event team please replay to
> > this mail and we will invite you to the online meeting.
> > Regards
> > Alin



-- 
Flavio de Castro Alves Filho

flavio.al...@gmail.com
Twitter: http://twitter.com/#!/fraviofii
LinkedIn profile: www.linkedin.com/in/flaviocastroalves

CI stuck?

[Matias N.]

I noticed CI wasn't running jobs and found several runs from 8 hrs ago which 
were stuck with "internal error" reports. I canceled those (event though there 
weren't actually running) to see if the queued ones were picked up (which 
doesn't seem to be the case). Anyone knows what could be happening?

Best,
Matias

Re: CI stuck?

[Brennan Ashton]

I saw this a few days ago when GitHub was having some issues and it looks
like they have an active outage now as well that is recovering so I would
suspect that.

https://www.githubstatus.com/

--Brennan

On Tue, May 18, 2021, 10:17 AM Matias N.  wrote:

> I noticed CI wasn't running jobs and found several runs from 8 hrs ago
> which were stuck with "internal error" reports. I canceled those (event
> though there weren't actually running) to see if the queued ones were
> picked up (which doesn't seem to be the case). Anyone knows what could be
> happening?
>
> Best,
> Matias

Re: CI stuck?

[Matias N.]

Thanks Brennan. So we will have to wait it out it seems.
In the meantime I looked around in the Apache builds list and found a thread 
from April
related to issues with GA. This thread included an analysis of which were the 
worst offenders
in CI time and we sadly came out on top. You can see a graph here as well:
https://cwiki.apache.org/confluence/display/BUILDS/GitHub+Actions+status

I'm hoping that with CMake we can really improve our build time (that is the 
impression
I get locally as well).

Best,
Matias

On Tue, May 18, 2021, at 15:10, Brennan Ashton wrote:
> I saw this a few days ago when GitHub was having some issues and it looks
> like they have an active outage now as well that is recovering so I would
> suspect that.
> 
> https://www.githubstatus.com/
> 
> --Brennan
> 
> On Tue, May 18, 2021, 10:17 AM Matias N.  > wrote:
> 
> > I noticed CI wasn't running jobs and found several runs from 8 hrs ago
> > which were stuck with "internal error" reports. I canceled those (event
> > though there weren't actually running) to see if the queued ones were
> > picked up (which doesn't seem to be the case). Anyone knows what could be
> > happening?
> >
> > Best,
> > Matias
> 

Re: CI stuck?

[Brennan Ashton]

Yeah I had tried to communicate that information here as well, this is why
I had suggested the cancellation jobs in the first place, when you enabled
(thank you!).  The graph is by workflow not job so it makes our usage look
a lot worse but yeah I agree we need to do what we can to improve our
resource usage not just for ourselves and I really appreciate the work you
have been putting into this!

--Brennan

On Tue, May 18, 2021, 11:14 AM Matias N.  wrote:

> Thanks Brennan. So we will have to wait it out it seems.
> In the meantime I looked around in the Apache builds list and found a
> thread from April
> related to issues with GA. This thread included an analysis of which were
> the worst offenders
> in CI time and we sadly came out on top. You can see a graph here as well:
> https://cwiki.apache.org/confluence/display/BUILDS/GitHub+Actions+status
>
> I'm hoping that with CMake we can really improve our build time (that is
> the impression
> I get locally as well).
>
> Best,
> Matias
>
> On Tue, May 18, 2021, at 15:10, Brennan Ashton wrote:
> > I saw this a few days ago when GitHub was having some issues and it looks
> > like they have an active outage now as well that is recovering so I would
> > suspect that.
> >
> > https://www.githubstatus.com/
> >
> > --Brennan
> >
> > On Tue, May 18, 2021, 10:17 AM Matias N.  matias%40imap.cc>> wrote:
> >
> > > I noticed CI wasn't running jobs and found several runs from 8 hrs ago
> > > which were stuck with "internal error" reports. I canceled those (event
> > > though there weren't actually running) to see if the queued ones were
> > > picked up (which doesn't seem to be the case). Anyone knows what could
> be
> > > happening?
> > >
> > > Best,
> > > Matias
> >
>

Re: Nuttx FAT32 issues - corrupted files, wrongly stated free clusters by statfs

[Alan Carvalho de Assis]

Hi Reto,

Mr. Jukka just opened a PR that could help the issue related to the
cluster issue:

https://github.com/apache/incubator-nuttx/pull/3740

When you face an issue with the SDCard a good idea is to do a raw copy
of it on Linux using the dd command, then you could duplicated the
issue later.

If you find an easy way to reproduce the issue, it will make it easier
for other people duplicate your issue and help to debug it.

BR,

Alan

On 5/18/21, Reto Gähwiler  wrote:
> Dear All,
>
> First of all, in case a similar thread pops-up authored by myself, please
> ignore.
>
> Recently we discovered some issues with the FAT32 partition on the SD-Card
> used in our device running on nuttx. There are actually a bunch of issues
> we are strugle to understand related to the filesystem.
> The issue discovered recently is, that a statfs call won't return the true
> number of free clusters. It rather returns what ever is in the FS INFO
> section of the FAT32 partition. Inserting such a "corrupted" card into an
> SD-Card reader and mounting it to Windows shows following:
>
> #1 Windows file explorer reports the correct free space.
> #2 Comparing the free space in file explorer and looging at FS INFO section
> with "Active - Disk Editor" doesn't line up. Windows wouldn't write any
> longer the FS INFO section on that card. Even if all the files/dires are
> wiped from the card.
> #3 Whenever files are added / removed under nuttx the FS INFO section would
> be updated by nuttx, but to the wrong number since the base is wrong.
> #4 Mounting the card to linux and properly unmount it might actually fix
> the issue. But not always!
> #5 Quick format the card resolved the issue too. Windows would once again
> write the FS INFO section and nuttx would shows the correct free space.
> #5 Running a "chkdsk /f" under windows on that broken SD-Card fixes the
> issue too. Reporting a broken file in a root folder on that card.
>
> We indeed had issues with corrupted files in the past. Usually in the
> logging directory, which is accessed most. How they corrupt we do not know.
> The corruption is recognised in the following ways:
>
> #A cryptic and too long file names, we only make use of short file names
> (8.3)
> #B files whcih became folders
> #C sizes which are not possible
> #D combinations of the above
>
> Our application supervises the SD-Card with statfs and removes files once
> we hit a quota of 0.85. Therefore, the card is not filling up. However, the
> most recent recognition of this behvaiour was due to an application bug
> which allowed the card to fill up under some circumstances.
> Playing around with a quickformated SD-Card, filled up with data and trying
> to retrigger the issue revealed that I never get the ENOSPC but only EIO
> once the card is full! But never managed to get the statfs issue.
> For the logging we use the fstream family (fopen, fwrite, fread, fflush,
> fsync) commands. For config files and other things also the low level
> read/write are used.
>
> One more thing, among all the logging files we once in a while have
> corrupted file content. Typically that is a skipped byte or doubled byte
> (we use protobuf and can see that encoding the binary). This seems to
> happen during reading but also writing the file.
>
> At the moment we didn't investigate the fat driver of nuttx but rather
> tried to analyse the problem. And now we have a bunch of question marks.
>
> #1 Has anyone recongised similar issues?
> #2 Does anyone know a tool which can be used in nuttx equivalent to chkdsk
> under windows? Or do quick format?
> #3 Why is there no ENOSPC but EIO once the card is full?
> #4 What triggers the corrupted files? How robust is the nuttx FAT32
> considering stackdumps and powerlosses?
>
> For the purpose of investigation, I have an image of the SD-Card when it
> was in the state of reporting wrong sizes in nuttx.
>
> Looking forward to your answers and ideas. Appreciate your help,
>
> Reto
>

Re: Nuttx FAT32 issues - corrupted files, wrongly stated free clusters by statfs

[Jukka Laitinen]

Hi,

There seems to be a bug in sector calculation which may trigger with 
e.g. corrupted fs. I just made a PR for this,


https://github.com/apache/incubator-nuttx/pull/3740

But I really don't know if this is related to your issues, this is just 
something that suddenly popped up elsewhere.


-Jukka


On 18.5.2021 16.10, Reto Gähwiler wrote:

Dear All,

First of all, in case a similar thread pops-up authored by myself, please
ignore.

Recently we discovered some issues with the FAT32 partition on the SD-Card
used in our device running on nuttx. There are actually a bunch of issues
we are strugle to understand related to the filesystem.
The issue discovered recently is, that a statfs call won't return the true
number of free clusters. It rather returns what ever is in the FS INFO
section of the FAT32 partition. Inserting such a "corrupted" card into an
SD-Card reader and mounting it to Windows shows following:

#1 Windows file explorer reports the correct free space.
#2 Comparing the free space in file explorer and looging at FS INFO section
with "Active - Disk Editor" doesn't line up. Windows wouldn't write any
longer the FS INFO section on that card. Even if all the files/dires are
wiped from the card.
#3 Whenever files are added / removed under nuttx the FS INFO section would
be updated by nuttx, but to the wrong number since the base is wrong.
#4 Mounting the card to linux and properly unmount it might actually fix
the issue. But not always!
#5 Quick format the card resolved the issue too. Windows would once again
write the FS INFO section and nuttx would shows the correct free space.
#5 Running a "chkdsk /f" under windows on that broken SD-Card fixes the
issue too. Reporting a broken file in a root folder on that card.

We indeed had issues with corrupted files in the past. Usually in the
logging directory, which is accessed most. How they corrupt we do not know.
The corruption is recognised in the following ways:

#A cryptic and too long file names, we only make use of short file names
(8.3)
#B files whcih became folders
#C sizes which are not possible
#D combinations of the above

Our application supervises the SD-Card with statfs and removes files once
we hit a quota of 0.85. Therefore, the card is not filling up. However, the
most recent recognition of this behvaiour was due to an application bug
which allowed the card to fill up under some circumstances.
Playing around with a quickformated SD-Card, filled up with data and trying
to retrigger the issue revealed that I never get the ENOSPC but only EIO
once the card is full! But never managed to get the statfs issue.
For the logging we use the fstream family (fopen, fwrite, fread, fflush,
fsync) commands. For config files and other things also the low level
read/write are used.

One more thing, among all the logging files we once in a while have
corrupted file content. Typically that is a skipped byte or doubled byte
(we use protobuf and can see that encoding the binary). This seems to
happen during reading but also writing the file.

At the moment we didn't investigate the fat driver of nuttx but rather
tried to analyse the problem. And now we have a bunch of question marks.

#1 Has anyone recongised similar issues?
#2 Does anyone know a tool which can be used in nuttx equivalent to chkdsk
under windows? Or do quick format?
#3 Why is there no ENOSPC but EIO once the card is full?
#4 What triggers the corrupted files? How robust is the nuttx FAT32
considering stackdumps and powerlosses?

For the purpose of investigation, I have an image of the SD-Card when it
was in the state of reporting wrong sizes in nuttx.

Looking forward to your answers and ideas. Appreciate your help,

Reto