On Fri, Mar 22, 2024 at 05:52:04PM +1100, Paul Colquhoun wrote
> Bash can do patern substitution in variable references.
>
> Replace accum3=$(( ${accum3} + ${dataarray[3]} ))
> with accum3=$(( ${accum3} + ${dataarray[3]/'.'/0} ))
>
> and similarly with the other lines and any array value of '.' will
> be replaced with a '0'
Replacing "." with a zero is not the problem. I already do this at
the top of the script with "sed"...
sed "s/\"\.\"/0/g" region_hospital_icu_covid_data.csv >
region_hospital_icu_covid_datax.csv
The problem is that a certain range of valid-looking zero sums in the
summary output is actually invalid and has to be zapped. I was
originally hoping to wipe that range with a macro after importing it
into gnumeric. "Plan B" is to tweak the way that a certain date range
gets written out. Note the difference starting at 2023-09-09...
2023-09-06,28,10,568,31,12,3,2
2023-09-07,30,6,594,33,8,3,2
2023-09-08,33,7,600,31,11,-2,4
2023-09-09,0,0,628,0,0,0,0
2023-09-10,0,0,657,0,0,0,0
2023-09-11,0,0,625,0,0,0,0
2023-09-06,28,10,568,31,12,3,2
2023-09-07,30,6,594,33,8,3,2
2023-09-08,33,7,600,31,11,-2,4
2023-09-09,,,628,0,0,0,0
2023-09-10,,,657,0,0,0,0
2023-09-11,,,625,0,0,0,0
Gnumeric imports it OK with nothing in the cells that have missing data
in the source. While this isn't the way I had originally intended, it
works, which is what matters. In the revised script, note the filter...
if [ "${prevdate}" \< "2023-09-09" ] || [ "${prevdate}" \> "2023-10-20" ]
One advantage of YYYY-MM-DD date format is that I can do straight
string comparisons. Here's the revised script...
==========================================================================
# Strip out missing "." that screw up the script
sed "s/\"\.\"/0/g" region_hospital_icu_covid_data.csv >
region_hospital_icu_covid_datax.csv
dos2unix -n region_hospital_icu_covid_datax.csv
region_hospital_icu_covid_datay.csv
#
# tail skips headers at beginning of file
# sed deletes Row_ID, and strips out quotes
# Output goes to file /dev/shm/temp0.txt
tail -n +2 region_hospital_icu_covid_datay.csv | sed "s/\"//g" | sort >
/dev/shm/temp0.txt
#
## Set up IFS for easier parsing
oldifs="${IFS}"
IFS=","
#
# Initialize previous line's date to enter loop smoothly
# expando to read first line
dataline=$( head -1 /dev/shm/temp0.txt )
dataarray=(${dataline})
prevdate="${dataarray[0]}"
#
# Zero out accumulators to enter loop smoothly
accum2=0
accum3=0
accum4=0
accum5=0
accum6=0
accum7=0
accum8=0
#
# Remove previous hospsum.csv and open a new one for writing
rm -rf hospsum.csv
exec 3>hospsum.csv
#
# Write header line to output file
echo
"date,icu_current_covid,icu_current_covid_vented,hospitalizations,icu_crci_total,icu_crci_total_vented,icu_former_covid,icu_former_covid_vented"
>&3
#
# Main loop
# Read the data from one line in /dev/shm/temp0.txt
while read
do
dataarray=(${REPLY})
if [ "${dataarray[0]}" = "${prevdate}" ]; then
#
# If this line's date is same as previous line's date, add amounts to
accumulators.
accum2=$(( ${accum2} + ${dataarray[2]} ))
accum3=$(( ${accum3} + ${dataarray[3]} ))
accum4=$(( ${accum4} + ${dataarray[4]} ))
accum5=$(( ${accum5} + ${dataarray[5]} ))
accum6=$(( ${accum6} + ${dataarray[6]} ))
accum7=$(( ${accum7} + ${dataarray[7]} ))
accum8=$(( ${accum8} + ${dataarray[8]} ))
else
#
# If this line's date has changed, output to hospsum.csv, update prevdate,
# and update accumulators. ***IMPORTANT*** "echo" TO hospsum.csv MUST BE
# EXECUTED BEFORE UPDATING ACCUMULATORS AND prevdate***
#
# Data *NOT* in range 2023-09-09 ... 2023-10-20 is written out in full.
# Data in that range gets null data for ${accum2} and ${accum3}
if [ "${prevdate}" \< "2023-09-09" ] || [ "${prevdate}" \> "2023-10-20" ]
then
echo
"${prevdate},${accum2},${accum3},${accum4},${accum5},${accum6},${accum7},${accum8}"
>&3
else
echo "${prevdate},,,${accum4},${accum5},${accum6},${accum7},${accum8}"
>&3
fi
prevdate="${dataarray[0]}"
accum2=${dataarray[2]}
accum3=${dataarray[3]}
accum4=${dataarray[4]}
accum5=${dataarray[5]}
accum6=${dataarray[6]}
accum7=${dataarray[7]}
accum8=${dataarray[8]}
fi
done</dev/shm/temp0.txt
#
# Close file hospsum.csv
exec 3<&-
IFS="${oldifs}"
--
Roses are red
Roses are blue
Depending on their velocity
Relative to you
