Dear all,
I am very happy to announce the 24th release (version 0.24) of
GNU Astronomy Utilities (Gnuastro). There have been 125 commits
by 11 people (see [1]) in the 71 weeks since version 0.23 (for
the most noteworthy updates see [2]).
Gnuastro is an official GNU package, consisting of various
command-line programs, C/C++ library functions and GNU Makefile
extensions for the manipulation and analysis of (astronomical)
data. All the programs share the same basic command-line user
interface (modeled on GNU Coreutils). See the Gnuastro webpage
for more details: https://www.gnu.org/s/gnuastro
Below are the links to download the compressed source for this
release in two formats. To uncompress Lzip tarballs, see [3]. To
check the validity of the tarballs using the GPG detached
signature (*.sig) see [4] and using SHA checksums see [5]. In [6]
you can see the list of software used to bootstrap this tarball.
https://ftp.gnu.org/gnu/gnuastro/gnuastro-0.24.tar.lz (4.9MB)
https://ftp.gnu.org/gnu/gnuastro/gnuastro-0.24.tar.gz (7.9MB)
A copy of the PDF manual and tarball of every release is also
placed on Zenodo to register a DOI for this release and help in
citations. For example see how the manual of Gnuastro 0.23 is
cited in https://arxiv.org/abs/2510.12940 . The DOI for version
0.24 is:
https://doi.org/10.5281/zenodo.17726900
If any of Gnuastro's programs or libraries are useful in your
work, please don't forget to cite the relevant publications in
your publications. For citation guidelines, run the relevant
programs with a `--cite' option (it can be different for
different programs, so run it for all the programs you use). For
the full list of Gnuastro's citable resources, see
https://ui.adsabs.harvard.edu/public-libraries/0QdYMuVCQdmygEh0Vs_4Ew
(requires Javascript).
Cheers,
Mohammad
--
https://akhlaghi.org
Staff researcher at CEFCA,
Lead of ARRAKIHS mission data reduction pipeline
[1] Thanks to everyone who has contributed!
The following people contributed changes to this release:
Barış Güngör (1)
Boud Roukema (1)
Faezeh Bidjarchian (2)
Giacomo Lorenzetti (13)
Haleh Mesgari (2)
Ignacio Ruiz Cejudo (1)
Mohammad Akhlaghi (92)
Rahna Payyasseri (1)
Raul Infante-Sainz (10)
Samane Raji (1)
Sepideh Eskandarlou (1)
The following people provided comments, suggestions or found
bugs:
Alejandro Camazón Pinilla
Alejandro Lumbreras-Calle
Antonio Diaz Diaz
Benson Muite
Carlos Marrero de La Rosa
David Valls-Gabaud
Elizabeth Sola
Faezeh Bidjarchian
Hok Kan (Ronald) Tsang
Joseph Putko
Ole Streicher
Phil Wyett
Rahna Payyasseri Thanduparackal
Rafael Guzman
Raul Infante-Sainz
Samane Raji
Sepideh Eskandarlou
[2] Noteworthy changes since version 0.23 (from the 'NEWS' file
within the tarball).
** New publications
- https://arxiv.org/abs/2510.12940 by Eskandarlou et al. that
describes the latest status of our scripts to extract and
subtract the extended PSF of bright stars in the J-PLUS
survey. See Figure 1 (for an outline of the flow chart) and
Figure 6 (on a nice demo of the effect). It builds upon
Infante-Sainz et al. 2020 (which was the basis for Gnuastro's
'astscript-psf-*' scripts) and like that paper, this one is
also fully reproducible so you can go inside its public
source code to check every step (see Section 6 for the
links).
** New features
*** Arithmetic
- New Operators: similar operators are grouped; groups are
sorted alphabetically by first operator in the group
- filter-minimum:
- filter-maximum:
- filter-sigclip-std:
- filter-madclip-mad:
- filter-sigclip-std:
- filter-madclip-mad:
Replace each pixel with the respective statistic of the
pixels in the given filter size. These are useful when you
need to measure the spread of nearby pixels (in the
presence of outliers).
- label-area:
- label-minimum:
- label-maximum:
Given a labeled and a values image do the requested
measurements on the label and set the label's pixels to the
output of the measurement in the output. For example, if
you want to remove all labels with an area less than 3, you
can use the 'label-area' option followed by the 'gt'
(greater-than) operator. Until now, to do this, we needed
to run MakeCatalog, extract the rows, and write loops to
remove the labels one by one (which was slow, in efficient
and could by buggy). These new operators greatly simply
such operations while significantly improving the
speed. See example usages within the book.
- luminosity-to-mag:
- mag-to-luminosity:
Convert apparent magnitude to luminosity and
vice-versa. Implemented by Sepideh Eskandarlou.
- madclip-all:
- sigclip-all:
Return all available statistics after clipping outliers
(mean, std, median and MAD and number of used datasets).
- sblim-diff: calculate the expected difference in surface
brightness limit (purely based on exposed area (of the
telescope) and exposure time (assuming the same filter, sky
conditions and assuming no correlated noise). This can be
used for planning projects with different telescopes, but
be careful about filters and atmosphere.
*** ConvertType
--cmapblankcolor: Name of color to use for blank values when
showing single-channel images through a colorbar. This is
very useful in scenarios where you don't want your audience
to confuse NaN pixels with pixels that have data. For example
if you use the Viridis colormap, you can set
'--cmapblankcolor=white' or '--cmapblankcolor=red' (which do
not exist in the Viridis color map). Note that this option
does not apply to the "gray" colormap because that is single
channel. To get the list of recognized colors in Gnuastro run
'astconvertt --listcolors'. This was suggested by Sepideh
Eskandarlou.
--cmappgfplots: create a second output file containing the
PGFPlots colormap definition for single-channel data that are
to be displayed through a colormap. PGFPlots is a powerful
plot creation package within LaTeX to create high-quality
plots and figures in your papers, slides or reports. With
this option, you can use Gnuastro colormaps that are not
available in PGFPlots. The "Annotations for figure in paper"
section of the book has a fully working example of the usage
of PGFPlots in combination with Gnuastro for high-quality
figure generation.
*** MakeCatalog
--novalinerror: ignore pixel in the values image when
calculating the variance of each label's pixel: only use the
pixel's value in the '--instd' image (which can also be the
variance with the '--variance' option). This is useful in
scenarios where the STD/variance image is not just the sky
STD/Variance, but also has the signal's contribution.
--mean-error: Measure the error of the mean for each label
(object or clump).
--meta-measures: new option to print metameasurements
(measurements on measurements) as keywords in the 0th HDU of
the output. These include the measured surface brightness
limit, the noise-based magnitude limit and the confusion
limit. Until now, only the first was always automatically
measured, but from this version, if you need to call this
option to get it (the keyword names have also changed, see
"MakeCatalog output keywords" section of the book). For more,
see the newly added section in the book called
"Metameasurements on full input".
--nml-sigma and --nml-area: the multiple of standard deviation
and area (in square arcseconds) to use for the noise-based
magnitude limit (also called "detection limit" by some
authors).
--cnl-with-objects: measure the confusion limit using objects
instead of clumps (only recommended if you are not generating
your labels with Segment).
--cnl-check: add an extra HDU to the output that contains the
distances to the nearest neighbor of each row used in the
confusion limit.
*** Match
- Ambiguous matches (multiple counterparts in other catalog
within aperture) will not be present in the main output. When
present, the ambiguous rows of the second input will be
listed in a dedicated HDU of the output (called
'FLAGGED-IN-2ND'). A dedicated new section (called
"Unambiguous matching") has been added to the book that
describes the intricacies of this new behavior.
--arrange: the arrangement of the match output. Until now, the
Match program only had the 'inner' arrangement (known in SQL
as "INNER JOIN"). From this version, other arrangements are
also available and fully described (with example tables and
short tutorials) in the newly added "Arranging match output"
section of the book. A short summary of the new arrangements
is given below:
- full: known as "FULL OUTER JOIN" in SQL. Its output has
both the matched rows and the non-matched rows in one
catalog.
- outer: known as "RIGHT OUTER JOIN" in SQL. Its output has
the same number of rows as the second (query) input catalog
to Match, giving the ability to find the (possibly
repeated) matching rows from the first (reference)
catalog. See the demo for a real-world example.
- outer-within-aperture: similar to 'outer', but if the
nearest reference catalog entry is more distant than the
given aperture, it will be NaN/blank in the output.
*** Statistics
- Polynomial fitting is now possible for 2D inputs: given either
as an image (assuming the pixel's horizontal and vertical
positions are the independent variables) or as a three columns
(where the first two are independent variables). A new "Two
dimensional polynomial fitting" section has been added in the
book to demonstrate its usage.
*** astscript-ds9-region
--fontsize: new option to allow setting the font size of the
values within the name column (given to '--namecol'). The
values of the name column are printed as text on top of each
region in DS9.
*** astscript-fits-view
--topcat4k (or '-k') option is added for easy display on 4K
monitors (where the TOPCAT window can be very small by
default).
*** astscript-radial-profile
--azimuth option can be called multiple times so the profile is
generated over discrete azimuthal regions of the input. For
example if called with '--azimuth=30,50 --azimuth=210,250',
the profile (or polar plot if requested) will be generated
only using pixels within these two azimuthal ranges (all
other pixels are ignored). Suggested and implemented by
Ignacio Ruiz Cejudo.
*** Makefile extensions
- $(ast-text-next TARGET, LIST): returns the next word (after
TARGET) in the input LIST; see the documentation for more.
- $(ast-text-next-words TARGET, NUM, LIST): returns the next
NUM words after TARGET from LIST.
*** Library
- gal_data_append_second_array_to_first_free: for easily
appending (adding the contents of the second array into the
first) in single-dimensional datasets.
- gal_dimension_image_to_table: function to extract all
non-blank elements of an image and save them into a table
with their pixel coordinates (first two columns) and pixel
value.
- GAL_FIT_MATRIX_INVALID:
- GAL_FIT_MATRIX_POLYNOMIAL_1D:
- GAL_FIT_MATRIX_POLYNOMIAL_2D:
- GAL_FIT_MATRIX_POLYNOMIAL_2D_TPV:
Macros to identify different types of fitting matrices.
- GAL_FIT_MATRIX_NUMBER_1D:
- GAL_FIT_MATRIX_NUMBER_ALL:
The number of 1D matrices and total matrices used in fitting.
- gal_fit_polynomial_tikhonov: new function to perform a
regularized fit using Tikhonov regression, wrapping over the
GNU Scientific Library (GSL) functions.
- gal_kdtree_range: returns the list of points that are within
a given radius of the given point. Implemented by Barış
Güngör.
- gal_label_measure: Replace a label with measured values over
it (see description of newly added 'label-*' operators in
Arithmetic).
- gal_list_sizetsizetf64_add:
- gal_list_sizetsizetf64_pop:
- gal_list_sizetsizetf64_free:
New functions to use a new list type that includes a 'size_t'
and a 'double'.
- gal_statistics_minimums_three: return the three smallest
values of the given dataset.
- gal_statistics_range_double: return the difference between
the minimum and maximum as a double-precision floating point
of a dataset (with any type).
- gal_tile_per_label: create an array/list of tiles that cover
the separate labels within the input.
- gal_table_sort: sort the given table by a given column.
- gal_units_sblim_diff: calculate the difference in surface
brightness limit only assuming a different effective radius
of a different telescope and/or a different exposure time.
- gal_units_mag_to_luminosity:
- gal_units_luminosity_to_mag:
Convert the observed magnitude of an object to luminosity and
vice-versa.
- gal_units_wavelength_flux_density_to_jy: convert given
wavelength flux density (erg/s/cm^2/A) to Janskys.
** Removed features
- Nothing has been removed in this release.
** Changed features
*** Book
- The term "stack" was used in two contexts within the
Arithmetic program: referring both to the stack of operands
during its operation and the coadding process (when we add
many images to make a deeper one). When talking about the
latter scenario, the book now uses the term "Coadd" (which is
also commonly used in the optical astronomical community).
- The sub-sections of MakeCatalog's old "Quantifying
measurement limits" section (which described individual
source errors as well as metameasures over the whole image)
have been distributed in other/new sub-sections following
MakeCatalog's new approach to metameasures as described above
in the new '--meta-measures' option.
*** All programs
--cite: also prints the citation for the Gnuastro book/manual
(using the Zenodo DOI for that release).
*** Arithmetic
- Operators:
- interpolate-minofregion:
- interpolate-maxofregion:
Now take three operands (in order of popping): the outer
connectivity, the inner connectivity and the image with
blank pixels. Until now, it took only the inner
connectivity and the outer connectivity was hard-coded to
the maximum of the input's dimensions. This was necessary
because it can happen that the most permissive outer
connectivity may be too loose in some usage scenarios. To
get the same output as before in your existing scripts, add
a '2' before the operator (for an image) or (3 for a cube).
- madclip-*: when the MAD is zero (median is repeated; mostly
occurs on integer inputs), an alternative to the MAD will
be used (difference between the two differing values just
before and after the median in a sorted array). This
greatly helps the usability of MAD-clipping on integer
datasets with small scatter (where the median can be
repeated a lot).
--append: when called with this option, Arithmetic will not
write its run-time option values in the 0-th HDU.
--metaname: can now accept multiple values (names of each HDU
for multi-output calls of Arithmetic).
--metaunit: similar to '--metaname'
--metacomment: similar to '--metaname'
*** Crop
- The log-file name (created when run with '--log') is based on
the output file name (see the description of '--log' under
the Crop program for the details) and is a FITS table with a
suffix '-log.fits'. Until now, the log file was always called
'astcrop.log' and it was created in the running
directory. This would cause problems for parallel
executions. Also, it was a plain-text file which is less
efficient: slower to read and write and would be larger for
longer catalog inputs.
*** MakeCatalog
- The output should always be a FITS file. This was necessary
because with the metameasures, more and more tables may be
requested by the user. Also, keywords are becoming more
important and and supporting both text and FITS was becoming
a major development overhead because plain-text doesn't
support multiple tables per file or keywords.
- All the metadata keywords of all the tables are now written
in the 0th HDU. Until now, they were written in all the
produced tables that was inefficient and confusing; this is
part of an already started project for all Gnuastro programs
to write all their metadata in the 0th HDU. The new
"MakeCatalog output keywords" section in the book fully
describes all the output keywords, please check that section
for all the new keyword names. The most significant is the
surface brightness limit keyword(s), that have now been
distributed and renamed to fit the new metameasure structure.
--sbl-sigma: new name for the old '--sfmagnsigma', as part of
the process to standardize metameasures.
--sbl-area: new name for the old '--sfmagarea', as part of the
process to standardize metameasures.
--forcereadstd has been removed in favor of the more general
'--meta-measure' option. This is because it was only useful
for calculating the surface brightness limit (which is a
metameasure).
*** Match
- Ambiguous matches (when more than one object is within the
given aperture) are flagged and kept in a new HDU of the
output. A new "Unambiguous matching" section has been added
in the "Match" section of the book to describe this important
feature.
- Output can only be in FITS format. Until this version, the
output could be plain-text also. However, plain-text tables
can only be a single table, so multiple files where necessary
when multiple tables were created. Managing all the files in
the various run-time scenarios was causing a significant
slow-down of the development and was also bugging (for
example '--logasoutput' which was not thread-safe). If a
plain-text format is necessary in your pipelines, you can
call Gnuastro's Table program to convert the given HDU into
plain-text.
--outcols: can now be invoked multiple times in one
command. Until this version it was only possible to call it
once, causing inconveniently long options for large
tables. This was suggested by Sepideh Eskandarlou.
*** astscript-ds9-region
- The default coordinate system to write WCS coordinates for
DS9 is now ICRS. Until now, it was FK5 (based on the default
system that DS9 would propose for writing regions). But DS9's
latest versions have changed to ICRS, so we are also doing
the same here. In practice, the difference is very small and
only relevant for extremely precise positioning.
*** astscript-pointing-simulate
--coadd-operator: new name for '--stack-operator'.
*** astscript-zeropoint
- The default matching radius for stars has been changed to 0.5
arcseconds. Until this version it was 0.2 arcseconds.
- If less than three stars are matched between the reference
and query catalogs, the script will abort with an elaborate
error message (guiding you on how to fix the problem. This is
because the result will not statistically reliable.
*** Library
- gal_blank_remove: new 'free_if_all_blank' to enable optional
freeing of the input array when all elements are blank.
- gal_kdtree_nearest_neighbor: new name of
'gal_kdtree_nearest_neighbour' which was in British English
(Gnuastro uses American English).
- gal_match_kdtree: a new 'arrange' and 'flag' argument have
been added to define the arrangement of the output.
- gal_match_sort_based: a new 'flag' argument has been added
that will be filled if there are ambiguous matches.
- gal_table_write_log: a new 'format' argument is added to
allow setting the format of the table (until now, it was
always plain-text).
- New names for the fitting library functions (old -->
new). This is because (as described in the new features), the
fitting functions have generalized to also accept 2D inputs.
- gal_fit_1d_linear --> gal_fit_linear_1d
- gal_fit_1d_polynomial --> gal_fit_polynomial
- gal_fit_1d_polynomial_robust --> gal_fit_polynomial_robust
- gal_fit_1d_linear_estimate --> gal_fit_linear_no_constant_1d
- gal_fit_1d_linear_no_constant --> gal_fit_linear_no_constant_1d
** Bugs fixed
- bug #66043: One pixel offset in PSF subtraction.
- bug #66216: NaN values in MakeProfile outputs for very small
profiles (smaller than a pixel even after accounting for the
truncation factor). Reported by Alejandro Camazón Pinilla.
- bug #66262: color-faint-gray crashes when using
--graykernelfwhm. Found and fixed by Samane Raji.
- bug #66280: Convolve's check images not in same directory as
output.
- bug #66303: astscript-color-faint-gray crashes if using 4th
channel. Reported by Elizabeth Sola and fixed by Raul
Infante-Sainz.
- bug #66308: Library's 'gal_wcs_img_to_world' and
'gal_wcs_world_to_img' returned inverse names when the
operation was not in-place. Found and fixed by Giacomo
Lorenzetti.
- bug #66309: Match crashes if kdtree is not set. Found and
fixed by Giacomo Lorenzetti.
- bug #66572: MakeProfiles continues adding pixels after
truncation when the profile is small (a couple of pixels),
and the gradient is strong.
- bug #66588: Arithmetic's 'trim' operator does not produce 0x0
image when all the image is blank.
- bug #66657: gal_fits_img_read not passing hdu_option_name;
found with the help of Sepideh Eskandarlou.
- bug #66693: MakeProfile crashes with segmentation fault with
when --customimg is given and the oversampling factor is
larger than 1.
- bug #66867: Example added in book to show how to remove
outputs of Query's '--overlapwith' option when sources
outside of the image are also retrieved (for example due to a
rotated image or the server not accounting for spherical
distortion). Reported by Raul Infante-Sainz.
- bug #66869: Crop segmentation fault leading to unpredictable
return values. Reported by Raul Infante-Sainz.
- bug #67105: astscript-radial-profile produced unreasonable
output when undersample was given as one. Reported by Samane
Raji and fixed by Raul Infante-Sainz.
- bug #67106: Crash during build when Ghostscript is not
present. Reported by Faezeh Bidjarchian.
- #67128: Maskfilled operators wrong output with zero scatter
(can happen in integer images, for example in the creation of
master dark frames).
- bug #67190: Sorting an empty plain-text table gives
segmentation fault. Reported by Sepideh Eskandarlou.
- bug #67191: Arithmetic does not parse '-inf' as a number on
the command-line and complains with an invalid '-i' option!
Reported by Sepideh Eskandarlou.
- bug #67204: Table's --equal operator cannot be called
multiple times on same column with different values.
- bug #67232: Building failure on i686 architectures (32-bit).
Reported by Benson Muite and fixed by Giacomo Lorenzetti.
- bug #67275: Segmentation fault in MakeCatalog during clump
catalog generation when input labels not contiguous (for
example you are feeding a crop of a larger object/clump map).
- bug #67297: Fits program segmentation fault when no input
specified and a HDU operation requested.
- bug #67364: Match's k-d tree based inner matching depends on
order of inputs. Fixed by Barış Güngör.
- bug #67475: Statistics fitting output prints 0 when the
values are very small. Reported by Faezeh Bidjarchian.
- bug #67684: Match's full-arrangement with outcols produces an
empty output when there is no match. Reported by Sepideh
Eskandarlou.
[3] Lzip has better compression ratio and archival features
compared to the '.gz' or '.xz' formats. Therefore Gnuastro's
alpha/test releases are only in this format, but for historical
reasons we also include `.gz' tarballs in the official
releases. If you don't have Lzip (you can check with `lzip
--version' command), download and install it from its webpage:
https://www.nongnu.org/lzip/lzip.html
If Lzip is present and you use GNU Tar, then the single command
below should uncompress and un-pack the tarball:
$ tar xf gnuastro-0.24.tar.lz
If the command above doesn't work, you have to un-compress and
un-pack it with two separate commands (or use a pipe to feed the
output of the first into the second: `lzip -cd
gnuastro-0.24.tar.lz | tar -xf -'):
$ lzip -d gnuastro-0.24.tar.lz
$ tar xf gnuastro-0.24.tar
[4] Use a .sig file to verify that the corresponding file
(without the .sig suffix) is intact. Here are the detached
signatures for the two formats above.
https://ftp.gnu.org/gnu/gnuastro/gnuastro-0.24.tar.gz.sig (833B)
https://ftp.gnu.org/gnu/gnuastro/gnuastro-0.24.tar.lz.sig (833B)
First, be sure to download both the .sig file and the
corresponding tarball. Then, run a command like this:
gpg --verify gnuastro-0.24.tar.lz.sig
The signature should match the fingerprint of the following key:
pub rsa4096 2018-12-08 [SC]
52B0 4484 D806 C90D CB52 7249 71E8 9901 2D17 4B66
uid [ unknown] Mohammad Akhlaghi <[email protected]>
If that command fails because you don't have the required public
key, or that public key has expired, try the following commands
to retrieve or refresh it, and then rerun the 'gpg --verify'
command.
gpg --recv-keys 71E899012D174B66
As a last resort to find the key, you can try the official GNU
keyring:
wget -q https://ftp.gnu.org/gnu/gnu-keyring.gpg
gpg --keyring gnu-keyring.gpg --verify gnuastro-0.24.tar.lz.sig
This release is based on the gnuastro git repository, available
as (with commit 586fa09014f790e2c0948035a375de276eac5163)
git clone https://https.git.savannah.gnu.org/git/gnuastro.git
[5] Here are the SHA256 and SHA3-256 checksums:
File: gnuastro-0.24.tar.gz
SHA256 sum:
e61c4a765e58816997c5882c5459945fc4b56294b3b77a965569309619aa8cdc
SHA3-256 sum:
e5512ab32590c7ebefc6e05293130705e543f4b605bf0f06999474745f3cf994
File: gnuastro-0.24.tar.lz
SHA256 sum:
150b1b566ecbcc07fe0cb55d32376b5b4488972274a6ad63d03298cda3c5e1bd
SHA3-256 sum:
f4f799b17ad2e768797e0d420906ef6e52c416ba82f991272f3b6fb6fb0ae893
Verify the SHA256 checksum with either sha256sum, sha256, or
shasum -a 256.
Verify the SHA3-256 checksum with cksum -a sha3 --check
from coreutils-9.8.
[6] This tarball was bootstrapped (created) with the tools
below. Note that you don't need these to build Gnuastro from the
tarball, these are the tools that were used to make the tarball
itself. They are only mentioned here to be able to
reproduce/recreate this tarball later.
Texinfo 7.2
Autoconf 2.72
Automake 1.18.1
Help2man 1.49.3
ImageMagick 7.1.2-8
Gnulib v1.0-2453-gbe6027b468
Autoconf archives v2023.02.20-111-gdcff834
The dependencies to build Gnuastro from this tarball on your
system are described here:
https://www.gnu.org/s/gnuastro/manual/html_node/Dependencies.html
