I just pushed the latest changes.
Functionality is identical.
Only minor improvements and bug fixes are included in the latest commit.
On Mon, Oct 30, 2023 at 12:06 PM Fotis Panagiotopoulos <f.j.pa...@gmail.com>
wrote:
> Hello Tim!
>
> Thank you for your interest in this.
>
> (I am replying in the list for the moment, in case anyone else is
> interested...)
>
> As far as I know, no effort has been taken on this.
> I will be glad to help you porting the settings storage to NuttX.
>
> The published repo is not maintained, so I will update it later today with
> all the latest
> changes/fixes in this specific app you are interested in.
>
> ---
>
> License-wise, this is 100% original work by me, so I guess there is
> nothing to concert us.
>
> It can be committed directly to the apps repo under Apache license.
>
> ---
>
> Regarding the parsers.
>
> Currently it supports two parsers ("storages" as they are refered in
> code): binary and text.
>
> The binary storage, directly dumps the RAM contents in a file.
> This is speed-efficient, but may waste a bit of space, and it is not very
> portable.
>
> For example, a save made in a little-endian machine may not work in a
> big-endian one.
> Similarly, the read and write implementations need to have the exact same
> cofniguration (e.g. key size).
>
> I typically use this to store settings directly in the MCU's Flash memory.
> So, none of the above limitations matter.
>
>
> The text storage solves all of the above and makes the data
> human-readable, in expense of some
> more CPU time.
>
> I typically use the text storage in an SD card file.
>
>
> More specifically, I usually use **both** storages at the same time.
> The text one is human-readable and easy to update (e.g. get the SD on your
> computer, and change anything you want),
> while the binary one is there for speed and reliability (SD card
> failed/removed, the app can still access the settings).
>
> The settings library can reliably syncrhonize multiple storages together,
> and make them act as one.
>
> ---
>
> My intention was to also add XML and JSON (and I have provided two parser
> implementations), but I never found the time.
> I guess adding YAML will be fairly trivial.
>
> I am willing to help add any new parsers that may be needed.
>
> ---
>
> So, shall we create an issue in apps repo, to cooperate development?
>
> On Mon, Oct 30, 2023 at 10:10 AM alin.jerpe...@sony.com <
> alin.jerpe...@sony.com> wrote:
>
>> Hi Tim,
>>
>> I am sorry but it slipped by agenda.
>> Thanks for looking at this topic. Please take your time and submit when
>> you are ready.
>>
>> Best regards
>> Alin
>>
>>
>> From: Tim Hardisty <timhardist...@gmail.com>
>> Sent: den 28 oktober 2023 12:10
>> To: dev@nuttx.apache.org
>> Subject: Re: Code donation
>>
>> Hi, Did anyone else make any progress with porting your settings code to
>> NuttX Apps? It's taken a while but I am at long last looking to add a
>> settings function to my new app and this still looks like it'll fit the
>> bill nicely. If not, I will
>> ZjQcmQRYFpfptBannerStart
>> This message is from an unknown sender
>> You have not previously corresponded with this sender and they are
>> external to Sony
>> ZjQcmQRYFpfptBannerEnd
>>
>> Hi,
>>
>>
>>
>> Did anyone else make any progress with porting your settings code to
>>
>> NuttX Apps? It's taken a while but I am at long last looking to add a
>>
>> settings function to my new app and this still looks like it'll fit the
>>
>> bill nicely.
>>
>>
>>
>> If not, I will set it up as an app, make sure it works, and submit it
>>
>> for review hopefully in the next few weeks. I won't add YAML as yet,
>>
>> just binary - my current think is to use a YAML parser between the user
>>
>> and this function rather than storing YAML directly, but not 100%
>>
>> decided yet.
>>
>>
>>
>> On 05/12/2022 11:22, Fotis Panagiotopoulos wrote:
>>
>> > Hello!
>>
>> >
>>
>> > Thank you for your interest in this!
>>
>> >
>>
>> > As requested, I created a temporal repository and uploaded all the code
>>
>> > there.
>>
>> >
>>
>> > https://github.com/fjpanag/code_for_nuttx<
>> https://github.com/fjpanag/code_for_nuttx>
>>
>> >
>>
>> > I have added a NOTICE file in each subdir, with some basic information.
>>
>> > Please contact me before starting any work, to provide you with help and
>>
>> > coordinate the effort.
>>
>> > Better yet, you may open a Github issue for the piece of software that
>> you
>>
>> > would like to help porting.
>>
>> >
>>
>> >
>>
>> > Alan,
>>
>> > I would love to cooperate with you in this effort.
>>
>> > Pick your target!
>>
>> >
>>
>> >
>>
>> > Alin,
>>
>> > I would really like to see my MQTT broker become part of NuttX.
>>
>> >
>>
>> > Alternatively, you can have a look at the XML or JSON parsers, which
>> are of
>>
>> > general usefulness,
>>
>> > and the porting effort would be very very minimal.
>>
>> >
>>
>> >
>>
>> > Tim,
>>
>> > Thank you for your interest in the settings storage.
>>
>> > It is a very useful and versatile piece of software. In fact, I have
>> used
>>
>> > it in all my projects over the last 5-6 years.
>>
>> >
>>
>> > I also thought about YAML before, but never got to actually implementing
>>
>> > this.
>>
>> >
>>
>> > I suggest you first add a YAML parser to NuttX (maybe similarly to JSON
>> and
>>
>> > XML, if they get accepted),
>>
>> > and then use it in the settings storage. It would be best not to couple
>>
>> > these two softwares so someone
>>
>> > may use one without the other. But, nevertheless, it is your call...
>>
>> >
>>
>> > If you provide me with a YAML parser, I believe that I can develop for
>> you
>>
>> > a new settings storage that uses this parser.
>>
>> >
>>
>> > Fotis
>>
>> >
>>
>> > On Mon, Dec 5, 2022 at 12:12 AM Tim Hardisty<t...@jti.uk.com
>> .invalid<mailto:t...@jti.uk.com.invalid>>
>>
>> > wrote:
>>
>> >
>>
>> >> I have interest in your settings storage, with the probability of
>> adding
>>
>> >> yaml output since I was going to do something similar for my current
>>
>> >> project (once I get over the pain of getting NuttX fixed for the arch
>> I'm
>>
>> >> using).
>>
>> >>
>>
>> >> As have been suggested, maybe push it to github and I/we can clone
>> what's
>>
>> >> of interest, see if it makes sense, then get it working right for NuttX
>>
>> >> and
>>
>> >> do a PR...in the fullness of time (i.e. I'm a slow worker!)
>>
>> >>
>>
>> >> On 04/12/2022, 16:55, "Fotis Panagiotopoulos"<f.j.pa...@gmail.com
>> <mailto:f.j.pa...@gmail.com>>
>>
>> >> wrote:
>>
>> >>
>>
>> >> Hello everyone!
>>
>> >>
>>
>> >> Christmas arrived a bit earlier for NuttX as I would like to
>> donate
>>
>> >> some of
>>
>> >> my personal code to the community!
>>
>> >>
>>
>> >> A bit of context.
>>
>> >> Over the years that I am working on embedded systems, I have
>> developed
>>
>> >> lots
>>
>> >> of software that I use in my projects.
>>
>> >> Some of it is quite general-purpose, or useful for other
>> applications,
>>
>> >> and
>>
>> >> I have found my self reusing it
>>
>> >> quite often. In fact, there are some things that I use in
>> practically
>>
>> >> all
>>
>> >> firmwares that I have developed over
>>
>> >> the last years.
>>
>> >>
>>
>> >> I always wanted to open-source this software so other people can
>>
>> >> benefit
>>
>> >> from it.
>>
>> >> But I never managed to do so. Open-sourcing needs some effort, the
>>
>> >> software
>>
>> >> needs maintenance, documentation
>>
>> >> and support, and most importantly in most cases a "porting layer"
>>
>> >> needs to
>>
>> >> be developed.
>>
>> >> Last but not least, every project needs a bit of "marketing" and
>>
>> >> "advertising" so others can learn about
>>
>> >> your work and use it.
>>
>> >>
>>
>> >> For the last couple of years I have been using NuttX a lot, and I
>> have
>>
>> >> ported most of the aforementioned software
>>
>> >> to NuttX. I believe that NuttX and its community are perfect for
>> me to
>>
>> >> publish my code, instead of creating
>>
>> >> a ton of small repos, of questionable usefulness and increasing my
>>
>> >> workload
>>
>> >> considerably.
>>
>> >>
>>
>> >> It is very important that I can get immediate feedback from the
>>
>> >> community,
>>
>> >> learn what people are actually
>>
>> >> interested in (instead of investing on software that no one
>> needs),
>>
>> >> and
>>
>> >> provide actual and *working*
>>
>> >> samples of the code (as NuttX already supports a ton of different
>>
>> >> boards
>>
>> >> and arches).
>>
>> >>
>>
>> >> Using POSIX as the porting layer is also awesome.
>>
>> >>
>>
>> >> That being said, my free time is still exceptionally limited and I
>>
>> >> cannot
>>
>> >> do this myself.
>>
>> >> I still need the help of the community, and most importantly I
>> need to
>>
>> >> see
>>
>> >> interest in a piece of
>>
>> >> software before putting any work on it.
>>
>> >>
>>
>> >> So, what I offer:
>>
>> >> * I offer various codes, fully featured, production ready and
>> tested.
>>
>> >> * All code will be offered for free (of course) and under Apache
>>
>> >> licensing.
>>
>> >> * I will provide support to those working on these codes, to my
>> best
>>
>> >> ability.
>>
>> >> * I will contribute to testing everything integrated to NuttX, as
>>
>> >> hardware
>>
>> >> availability allows me.
>>
>> >> * I will do some licensing check, to ensure code is 100% original
>> and
>>
>> >> mine,
>>
>> >> or state the licenses of the projects I borrowed code from.
>>
>> >>
>>
>> >> What I ask for:
>>
>> >> I need people that are interested in each of these codes to
>> integrate
>>
>> >> them
>>
>> >> into NuttX apps.
>>
>> >> You just have to pick what it is interesting to you, contact me to
>>
>> >> provide
>>
>> >> you with the code,
>>
>> >> and integrate it to NuttX. You will need to:
>>
>> >> * Add the code into the NuttX apps repo, and ensure Kconfig and
>> the
>>
>> >> build
>>
>> >> system use the code properly (should be trivial).
>>
>> >> * Adapt the file format and the coding style to the NuttX one
>> (this
>>
>> >> may
>>
>> >> need some work, but it can also be automated).
>>
>> >> * Provide an example app, something that someone can run to use or
>>
>> >> demo the
>>
>> >> new code.
>>
>> >> * Test and verify the example app on actual hardware (I may be
>> able to
>>
>> >> cross-check it on my hardware too).
>>
>> >>
>>
>> >> The code that I offer (for the moment):
>>
>> >>
>>
>> >>
>>
>> >> *** Lua v5.2.4 ***
>>
>> >> I know that there is already a Lua app for NuttX.
>>
>> >> But for anyone using it, it may be beneficial to use my work.
>>
>> >>
>>
>> >> First and foremost, I have ported the eLua LTR patch to Lua 5.2.
>> This
>>
>> >> patch
>>
>> >> dramatically reduces the memory usage of Lua.
>>
>> >> In fact, I found out that it is crucial to have this patch
>> enabled for
>>
>> >> any
>>
>> >> actual real-life usage of Lua on any "normal" MCU.
>>
>> >>
>>
>> >> I have created a Kconfig for all Lua configurations, so it can
>>
>> >> integrate
>>
>> >> with NuttX better.
>>
>> >>
>>
>> >> I have also made some other minor changes to the code that might
>> be
>>
>> >> interesting for you.
>>
>> >> For example there is a simplistic sandboxing option.
>>
>> >>
>>
>> >>
>>
>> >> *** MQTT Broker ***
>>
>> >> Yes, a full-blown, spec-compliant MQTT Broker!
>>
>> >> To my knowledge there is no other open-source and portable MQTT
>> broker
>>
>> >> for
>>
>> >> embedded systems.
>>
>> >>
>>
>> >> It follows the MQTT v3.1.1 specification as closely as possible. I
>>
>> >> think
>>
>> >> there is only one violation, needed due to its embedded nature,
>>
>> >> but in all practical cases you may consider it fully compliant.
>>
>> >>
>>
>> >> It has been tested with dozens of devices, and it performs
>> greatly.
>>
>> >> There are a couple of things that may need to be improved, but are
>>
>> >> trivial,
>>
>> >> and will not affect the normal use of the software.
>>
>> >>
>>
>> >> I know that such a broker may not be your best option for a
>> proper and
>>
>> >> large installation of IoT devices, but it is exceptionally useful
>>
>> >> for at least the following cases:
>>
>> >> * You have only a few devices, isolated (no internet), that you
>> need
>>
>> >> to
>>
>> >> connect, and you want to avoid the cost (and maintenance) of a
>> proper
>>
>> >> broker (e.g. Raspberry Pi).
>>
>> >> * You need to directly communicate with a device that only
>> supports
>>
>> >> MQTT.
>>
>> >> Instead of going through an external server, you run the server
>>
>> >> locally,
>>
>> >> and communicate with the device directly.
>>
>> >>
>>
>> >>
>>
>> >> *** MQTT Client ***
>>
>> >> A production-tested, robust and quite flexible MQTT client.
>>
>> >>
>>
>> >> I know that there are plenty of such clients available out there,
>> but
>>
>> >> here
>>
>> >> is another one.
>>
>> >>
>>
>> >> Back in the day I tried to use the Eclipse Paho library. I found
>> it to
>>
>> >> be a
>>
>> >> horrible piece of software. Crashes, buffer overflows, spec
>>
>> >> violations,
>>
>> >> missing functionality and more.
>>
>> >> I don't know whether it has improved now, but back then I
>> rolled-out
>>
>> >> my own
>>
>> >> implementation. There were not many other alternatives available,
>> and
>>
>> >> Paho
>>
>> >> did not worth contributing to it.
>>
>> >> It needed so much work, that starting from scratch was much
>> easier.
>>
>> >>
>>
>> >> Then, when I started using NuttX, I saw support for MQTT-C. Well,
>> I
>>
>> >> tried
>>
>> >> it and I wasn't greatly satisfied (I don't remember the reasons).
>>
>> >> So I decided to keep using my own (and well tested)
>> implementation.
>>
>> >>
>>
>> >> So NuttX, instead of relying on an external project, now can have
>> its
>>
>> >> own
>>
>> >> client.
>>
>> >>
>>
>> >>
>>
>> >> *** Settings Storage ***
>>
>> >> This is probably my favorite.
>>
>> >> A key-value pairs storage for non-volatile settings or
>> configurations.
>>
>> >> Very needed when you need to adjust the system's operation in
>>
>> >> run-time,
>>
>> >> download parameters etc.
>>
>> >>
>>
>> >> It acts in two layers: the settings API and the actual storages
>> used.
>>
>> >>
>>
>> >> The settings can store or retrieve key-value pairs.
>>
>> >> Type casts are supported and are performed automatically. For
>> example
>>
>> >> you
>>
>> >> can store the string "true" and then read it as the int 1 etc.
>>
>> >> There is support for caching writes (minimizing overhead and
>> reducing
>>
>> >> wear
>>
>> >> on the actual storage medium).
>>
>> >> There are signal notifications when a setting value is changed.
>>
>> >>
>>
>> >> The bottom layer, the storages, is responsible for actually
>> reading
>>
>> >> and
>>
>> >> writing the settings map to the physical mediums.
>>
>> >> Every storage can also format the data according to its type.
>>
>> >> For the moment there are only ASCII and binary storages, but it
>> would
>>
>> >> be
>>
>> >> very easy to expand this to JSON, XML, YAML and more.
>>
>> >> There is support for multiple storages used simultaneously, and
>>
>> >> synchronization between them (for example I usually use both the
>> MCU's
>>
>> >> Flash and the SD card).
>>
>> >>
>>
>> >>
>>
>> >> *** FTP Client ***
>>
>> >> This is a client that I had developed before I learned about
>> NuttX.
>>
>> >> I also tried NuttX FTPc, but I experienced may failures that I
>> never
>>
>> >> got
>>
>> >> the time to troubleshoot. I just went back to the tried solution.
>>
>> >> One thing that I didn't really like in NuttX ftpc, it was that it
>> is
>>
>> >> very
>>
>> >> taciturn. When something went wrong, you couldn't understand what
>> the
>>
>> >> issue
>>
>> >> is.
>>
>> >> The error information of this library is a bit better.
>>
>> >>
>>
>> >> If an alternative ftpc implementation is of interest to anyone,
>> the
>>
>> >> code is
>>
>> >> available.
>>
>> >>
>>
>> >>
>>
>> >> *** JSON Parser ***
>>
>> >> This is a very minimal parse-only JSON implementation.
>>
>> >>
>>
>> >> The motivation behind developing this was memory consumption.
>>
>> >> This parser can open an arbitrarily large JSON file (even MBs),
>> using
>>
>> >> only
>>
>> >> a few bytes of RAM!
>>
>> >> (If I recall correctly only 20 bytes!)
>>
>> >>
>>
>> >> Although there are other options there for you, if you need the
>>
>> >> smallest
>>
>> >> possible memory footprint, this is your parser.
>>
>> >>
>>
>> >>
>>
>> >> *** XML Parser ***
>>
>> >> Exactly as above with JSON.
>>
>> >> A minimal parser, that can open any file using just a few bytes of
>>
>> >> RAM.
>>
>> >>
>>
>> >> If you need something simple or you have memory constraints, use
>> this.
>>
>> >>
>>
>> >>
>>
>> >> *** XTEA ***
>>
>> >> Simple XTEA encryption and decryption.
>>
>> >> Very useful when you need a quite fast way to encrypt (or obscure)
>>
>> >> some
>>
>> >> data.
>>
>> >> (Need to check licensing!)
>>
>> >>
>>
>> >>
>>
>> >> *** Sun Calculations ***
>>
>> >> A small library that can do astronomical calculations for the sun.
>>
>> >> That is sunrise time, sunset, day duration and more.
>>
>> >>
>>
>> >>
>>
>> >> *** Geolocation ***
>>
>> >> I used the ipgeolocation.io API and NuttX wget to get geolocation
>>
>> >> information about the device.
>>
>> >> Nothing special, but it may serve as a working example, or even
>>
>> >> inspiration
>>
>> >> for others...
>>
>> >>
>>
>> >>
>>
>> >> If everyone is interested in any of the above please contact me.
>>
>> >>
>>
>> >> *PLEASE be responsible, and respectful.* (I am sure you will! :D )
>>
>> >> If you request for a piece of software* you will have to help to
>> get
>>
>> >> this
>>
>> >> into NuttX, and not only use it for your own personal gain*.
>>
>> >>
>>
>> >>
>>
>> >>
>>
>
