A podcast featuring panelists of engineers from Netflix, Twitch, & Atlassian talking over drinks about all things software engineering.
…
continue reading
Innehåll tillhandahållet av HPR Volunteer and Hacker Public Radio. Allt poddinnehåll inklusive avsnitt, grafik och podcastbeskrivningar laddas upp och tillhandahålls direkt av HPR Volunteer and Hacker Public Radio eller deras podcastplattformspartner. Om du tror att någon använder ditt upphovsrättsskyddade verk utan din tillåtelse kan du följa processen som beskrivs här https://sv.player.fm/legal.
Player FM - Podcast-app
Gå offline med appen Player FM !
Gå offline med appen Player FM !
HPR4201: Today I learnt (2024-08-23)
MP3•Episod hem
Manage episode 438855099 series 44008
Innehåll tillhandahållet av HPR Volunteer and Hacker Public Radio. Allt poddinnehåll inklusive avsnitt, grafik och podcastbeskrivningar laddas upp och tillhandahålls direkt av HPR Volunteer and Hacker Public Radio eller deras podcastplattformspartner. Om du tror att någon använder ditt upphovsrättsskyddade verk utan din tillåtelse kan du följa processen som beskrivs här https://sv.player.fm/legal.
TIL 1: Incrementing times with the date
command
- I have been working on an HPR project which is restoring external files to shows which lost them when we migrated to the current static site.
- As I make changes I want to be able to check that they are correct, but to make this check I need to wait for the next update of the static site.
- Ken was away recently, and set up a
cron
job to refresh the site every three hours. Each show page shows the refresh time in UTC form in the header. - Being a bit numerically challenged I wanted a way of computing the next refresh time in my timezone from the previous refresh time.
- The GNU
date
command accepts a date and time expression after the-d
option (or using the alternative--date=STRING
option). The contents of theSTRING
here are very flexible but quite complex since you can include time zone data, offsets, day and month names, etc. See the links below for links to the GNU manual. - My first attempt used the
date
command like this and got the wrong answer (using the output format+%T
which writes the time in a default form):
$ date -d '16:27:16 + 3 hours' +%T 15:27:16
- It is not clear why this fails, but the GNU function which parses these date parameters is obviously confused. The second try included the time zone after the time, and worked better, but is a little confusing:
$ date -d '16:27:16 UTC + 3 hours' +%T 20:27:16
- The time returned is local time for me. The
date
command has added three hours to the UTC date to get 19:27:16, but since I am in the UK, which is in DST (called BST - British Summer Time - UTC plus 1 hour), an hour is added. - The final try used the
-u
option which writes UTC time:
$ date -u -d '16:27:16 UTC + 3 hours' +%T 19:27:16
- I actually ended up using and re-using these commands (though a script would have been better):
$ current='06:27:55' $ next=$(date -u -d "${current}UTC + 3 hours 3 minutes" +%T); echo "$next UTC / $(date -d "${next} UTC" +'%T %Z')" 09:30:55 UTC / 10:30:55 BST $ current=$next $ next=$(date -u -d "${current}UTC + 3 hours 3 minutes" +%T); echo "$next UTC / $(date -d "${next} UTC" +'%T %Z')" 12:33:55 UTC / 13:33:55 BST $ current=$next $ next=$(date -u -d "${current}UTC + 3 hours 3 minutes" +%T); echo "$next UTC / $(date -d "${next} UTC" +'%T %Z')" 15:36:55 UTC / 16:36:55 BST
TIL 1: Links
- GNU Coreutils manual
- System context
- date: Print or set system date and time
- Date input formats (The quotation at the start of this section of the manual is quite interesting!)
- Date examples
- date: Print or set system date and time
- System context
TIL 2: Merging lines of files with paste
- While processing and "repairing" shows I came across the need to generate a list of show numbers separated by commas. In the past I have loaded these into a Bash array and turned them into a comma-delimited string, using the parameter substitution capabilities of Bash which can add a comma to each element. The trouble with this is that it leaves a trailing comma which has to be removed. I stumbled upon
paste
as an alternative way of doing this. - The GNU
paste
command is another from the GNU Coreutils group. This one merges lines of files. Its synopsis is:
paste [OPTION]... [FILE]...
- It merges lines consisting of the corresponding lines from each file provided as an argument, by default separated by TABs, and writes them to standard output. This means it produces lines consisting of the first line from each of the files, separated by tabs, then the second lines, and so on.
- Any of the files can be linked to standard in by using a
-
(hyphen) as the file name. - The delimiters can be changed with the
-d LIST
or--delimiters=LIST
option. The use of a list of delimiters causes the characters in the list to be used sequentially for each delimiter. - The merged lines can be visualised as rows in a matrix, where each file provides a column.
- The "matrix" is rotated by using the
-s
or--serial
option. Here the lines from one file at a time are merged - the files are processed serially rather than in parallel. - The
paste
command can be used to generate the comma-delimited list I wanted by using the options-s
and-d ','
:
$ printf '%s\n' {1..10} | paste -s -d, - 1,2,3,4,5,6,7,8,9,10
- Note that you can't get the same result with
echo {1..10}
because all the numbers will be written to one line rather than being the separate lines thatpaste
requires. - The file arguments to
paste
may also be Bash process substitution expressions:
$ paste -s -d, <(printf '%s\n' {1..10}) 1,2,3,4,5,6,7,8,9,10
- This means you can generate more complex output by using multiple process substitution expressions where each is seen as a file:
$ paste -d'|' <(printf '%d\n' {1..7}) <(printf '%s\n' {A..G}) <(printf '%d\n' {100..106}) 1|A|100 2|B|101 3|C|102 4|D|103 5|E|104 6|F|105 7|G|106
- Note how using the
-s
option "rotates" this:
$ paste -s -d'|' <(printf '%d\n' {1..7}) <(printf '%s\n' {A..G}) <(printf '%d\n' {100..106}) 1|2|3|4|5|6|7 A|B|C|D|E|F|G 100|101|102|103|104|105|106
- This is what I was actually trying to do, so I could feed show numbers to another script which will accept a CSV list as an option:
$ echo "select episode_id from repairs where repair_date is null order by episode_id desc limit 5" |\ sqlite3 -list ~/HPR/InternetArchive/ia.db | paste -s -d',' - 1959,1952,1951,1946,1941
TIL 2: Links
113 episoder
MP3•Episod hem
Manage episode 438855099 series 44008
Innehåll tillhandahållet av HPR Volunteer and Hacker Public Radio. Allt poddinnehåll inklusive avsnitt, grafik och podcastbeskrivningar laddas upp och tillhandahålls direkt av HPR Volunteer and Hacker Public Radio eller deras podcastplattformspartner. Om du tror att någon använder ditt upphovsrättsskyddade verk utan din tillåtelse kan du följa processen som beskrivs här https://sv.player.fm/legal.
TIL 1: Incrementing times with the date
command
- I have been working on an HPR project which is restoring external files to shows which lost them when we migrated to the current static site.
- As I make changes I want to be able to check that they are correct, but to make this check I need to wait for the next update of the static site.
- Ken was away recently, and set up a
cron
job to refresh the site every three hours. Each show page shows the refresh time in UTC form in the header. - Being a bit numerically challenged I wanted a way of computing the next refresh time in my timezone from the previous refresh time.
- The GNU
date
command accepts a date and time expression after the-d
option (or using the alternative--date=STRING
option). The contents of theSTRING
here are very flexible but quite complex since you can include time zone data, offsets, day and month names, etc. See the links below for links to the GNU manual. - My first attempt used the
date
command like this and got the wrong answer (using the output format+%T
which writes the time in a default form):
$ date -d '16:27:16 + 3 hours' +%T 15:27:16
- It is not clear why this fails, but the GNU function which parses these date parameters is obviously confused. The second try included the time zone after the time, and worked better, but is a little confusing:
$ date -d '16:27:16 UTC + 3 hours' +%T 20:27:16
- The time returned is local time for me. The
date
command has added three hours to the UTC date to get 19:27:16, but since I am in the UK, which is in DST (called BST - British Summer Time - UTC plus 1 hour), an hour is added. - The final try used the
-u
option which writes UTC time:
$ date -u -d '16:27:16 UTC + 3 hours' +%T 19:27:16
- I actually ended up using and re-using these commands (though a script would have been better):
$ current='06:27:55' $ next=$(date -u -d "${current}UTC + 3 hours 3 minutes" +%T); echo "$next UTC / $(date -d "${next} UTC" +'%T %Z')" 09:30:55 UTC / 10:30:55 BST $ current=$next $ next=$(date -u -d "${current}UTC + 3 hours 3 minutes" +%T); echo "$next UTC / $(date -d "${next} UTC" +'%T %Z')" 12:33:55 UTC / 13:33:55 BST $ current=$next $ next=$(date -u -d "${current}UTC + 3 hours 3 minutes" +%T); echo "$next UTC / $(date -d "${next} UTC" +'%T %Z')" 15:36:55 UTC / 16:36:55 BST
TIL 1: Links
- GNU Coreutils manual
- System context
- date: Print or set system date and time
- Date input formats (The quotation at the start of this section of the manual is quite interesting!)
- Date examples
- date: Print or set system date and time
- System context
TIL 2: Merging lines of files with paste
- While processing and "repairing" shows I came across the need to generate a list of show numbers separated by commas. In the past I have loaded these into a Bash array and turned them into a comma-delimited string, using the parameter substitution capabilities of Bash which can add a comma to each element. The trouble with this is that it leaves a trailing comma which has to be removed. I stumbled upon
paste
as an alternative way of doing this. - The GNU
paste
command is another from the GNU Coreutils group. This one merges lines of files. Its synopsis is:
paste [OPTION]... [FILE]...
- It merges lines consisting of the corresponding lines from each file provided as an argument, by default separated by TABs, and writes them to standard output. This means it produces lines consisting of the first line from each of the files, separated by tabs, then the second lines, and so on.
- Any of the files can be linked to standard in by using a
-
(hyphen) as the file name. - The delimiters can be changed with the
-d LIST
or--delimiters=LIST
option. The use of a list of delimiters causes the characters in the list to be used sequentially for each delimiter. - The merged lines can be visualised as rows in a matrix, where each file provides a column.
- The "matrix" is rotated by using the
-s
or--serial
option. Here the lines from one file at a time are merged - the files are processed serially rather than in parallel. - The
paste
command can be used to generate the comma-delimited list I wanted by using the options-s
and-d ','
:
$ printf '%s\n' {1..10} | paste -s -d, - 1,2,3,4,5,6,7,8,9,10
- Note that you can't get the same result with
echo {1..10}
because all the numbers will be written to one line rather than being the separate lines thatpaste
requires. - The file arguments to
paste
may also be Bash process substitution expressions:
$ paste -s -d, <(printf '%s\n' {1..10}) 1,2,3,4,5,6,7,8,9,10
- This means you can generate more complex output by using multiple process substitution expressions where each is seen as a file:
$ paste -d'|' <(printf '%d\n' {1..7}) <(printf '%s\n' {A..G}) <(printf '%d\n' {100..106}) 1|A|100 2|B|101 3|C|102 4|D|103 5|E|104 6|F|105 7|G|106
- Note how using the
-s
option "rotates" this:
$ paste -s -d'|' <(printf '%d\n' {1..7}) <(printf '%s\n' {A..G}) <(printf '%d\n' {100..106}) 1|2|3|4|5|6|7 A|B|C|D|E|F|G 100|101|102|103|104|105|106
- This is what I was actually trying to do, so I could feed show numbers to another script which will accept a CSV list as an option:
$ echo "select episode_id from repairs where repair_date is null order by episode_id desc limit 5" |\ sqlite3 -list ~/HPR/InternetArchive/ia.db | paste -s -d',' - 1959,1952,1951,1946,1941
TIL 2: Links
113 episoder
כל הפרקים
×Välkommen till Player FM
Player FM scannar webben för högkvalitativa podcasts för dig att njuta av nu direkt. Den är den bästa podcast-appen och den fungerar med Android, Iphone och webben. Bli medlem för att synka prenumerationer mellan enheter.