Terminal Tricks

In this article I will be collecting usful tricks and tweaks with Terminal on macOS. Some or even most of them could run on other UNIX systems as well.

jq

If we have a json file where KeyMetadata element has a KeyId

... | jq -r '.KeyMetadata.KeyId' # gives
111111-111111-11111-111111

... | jq '.KeyMetadata.KeyId'
"111111-111111-11111-111111"

jq '.[] | .login, .id'

fzf

It’s called an interactive grep.

One of the best usages for fzf would be to search through logs on a remote machine. At least, that what I’ve been learning to use it for a while. An example below utilises fzf with jq:

# timelimit with UNIX timestamps
aws cloudtrail lookup-events --profile ProfileName --start-time 1655535200 --end-time 1655542800 | jq -c '.Events[]' | fzf --multi --cycle --reverse --preview-window=right:80%:wrap --preview 'echo {} | jq .'

⛔ïļ When trying to parse some responses from Bitbucket API I’ve run into issue with fzf preview. ✍ðŸŧ Using echo -E {} instead of just echo {} has helped. This option turns off the interpretation of backslash-escaped characters on systems where this mode is the default. So, the problem was there I guess.

fx

Only works interactively with json, not ndjson. It’s good for long json file that has no repeated elements, since it’s interactive. Check the demo gif.

dasel

Converts data into various formats. For example, json into csv and vica versa.

jq '.' sheet1.json | dasel -r json -w csv > 10_04-05.csv # open sheet1.json with jq, then pipe to dasel, read json, write csv, save output to 10_04-05.csv.

Link here - https://github.com/TomWright/dasel. Might be iseful for some log analysis or when preparing some test tasks.

dsq

Query json, ndjson, csv, Excel, Parquet etc with SQL queries:

dsq testdata.json "SELECT * FROM {} WHERE x > 10"
dsq testdata/userdata.parquet 'select count(*) from {}' | jq # to beautify json

Link to docs.

python

python3 -m http.server # quickly share files across loal network 

References

Expand … [1] How do I select multiple fields in jq? gron: Make JSON greppable - https://github.com/tomnomnom/gron jello: Filter JSON and JSON Lines data with Python syntax - https://github.com/kellyjonbrazil/jello jless: Command-line pager for JSON data - https://github.com/PaulJuliusMartinez/jless jid: Json incremental digger - https://github.com/simeji/jid jql: JSON query language CLI tool - https://github.com/yamafaktory/jql qp: Command-line (ND)JSON querying - https://github.com/paybase/qp [0]: https://github.com/multiprocessio/datastation vd: A terminal spreadsheet multitool for discovering and arranging data -https://github.com/saulpw/visidata (edited) https://github.com/multiprocessio/datastation https://github.com/cado-security/varc https://github.com/multiprocessio/datastation