2626< section >
2727< h1 id ="datatools "> datatools</ h1 >
2828< p > < em > datatools</ em > is a rich collection of command line programs
29- targetting data conversion, cleanup and analysis directly from your
30- favorite POSIX shell. It has proven useful for data collaborations where
31- individual members of a project may prefer different toolsets in their
32- analysis (e.g. Julia, R, Python) but want to work from a common
33- baseline. It also has been used intensively for internal reporting from
34- various Caltech Library metadata sources.</ p >
29+ targeting data conversion, cleanup and analysis directly from your
30+ favorite POSIX shell or Powershell . It has proven useful for data
31+ collaborations where individual members of a project may prefer
32+ different tool sets in their analysis (e.g. Julia, R, Python) but want
33+ to work from a common baseline. It also has been used intensively for
34+ internal reporting from various Caltech Library metadata sources.</ p >
3535< p > The tools fall into three broad categories</ p >
3636< ul >
3737< li > data transformation and conversion</ li >
@@ -41,28 +41,27 @@ <h1 id="datatools">datatools</h1>
4141</ ul >
4242< p > See < a href ="user-manual.html "> user manual</ a > for a complete list of
4343the command line programs. The data transformation tools include support
44- for formats such as Excel XML, csv, tab delimited files, json, yaml and
45- toml.</ p >
44+ for formats such as Excel XML, csv, tab delimited files, json, yaml,
45+ toml and url encoding/decoding .</ p >
4646< p > Compiled versions of the datatools collection are provided for Linux
47- (amd64), Mac OS X (amd64), Windows 10 (amd64) and Raspbian (ARM7). See
47+ (aarch64/amd64), Mac OS X (aarch64/amd64), Windows 10 (aarch64/amd64)
48+ and Raspberry Pi OS (aarch64). See
4849https://github.com/caltechlibrary/datatools/releases.</ p >
4950< p > Use “-help” option for a full list of options for each utility
5051(e.g. < code > csv2json -help</ code > ).</ p >
5152< h2 id ="data-transformation "> Data transformation</ h2 >
5253< p > The tooling around transformation includes data conversion. These
53- include tools that work with CSV, tab delimited, JSON, TOML, YAML and
54- Excel XML.</ p >
54+ include tools that work with CSV, tab delimited, JSON, TOML, YAML, Excel
55+ XML, and url encoded text .</ p >
5556< p > There is also tooling to change data shapes using JSON as the
5657intermediate data format.</ p >
5758< h2 id ="for-the-shell "> For the shell</ h2 >
5859< p > Various utilities for simplifying work on the command line.</ p >
5960< ul >
60- < li > < a href ="docs/findfile/ "> findfile</ a > - find files based on prefix,
61- suffix or contained string</ li >
62- < li > < a href ="docs/finddir/ "> finddir</ a > - find directories based on
63- prefix, suffix or contained string</ li >
6461< li > < a href ="docs/mergepath/ "> mergepath</ a > - prefix, append, clip path
6562variables</ li >
63+ < li > < a href ="reldocpath.1.html "> reldocpath</ a > - calculates relative
64+ paths given do paths</ li >
6665< li > < a href ="docs/range/ "> range</ a > - emit a range of integers (useful
6766for numbered loops in Bash)</ li >
6867< li > < a href ="docs/reldate/ "> reldate</ a > - display a relative date in
@@ -78,7 +77,7 @@ <h2 id="for-strings">For strings</h2>
7877command for working with text strings (limited to memory available).
7978This is commonly needed when cleanup data for analysis. The
8079< em > string</ em > command was created for when the old Unix standbys-
81- grep, awk, sed, tr are unwieldly or inconvenient. < em > string</ em >
80+ grep, awk, sed, tr are unwieldy or inconvenient. < em > string</ em >
8281provides operations are common in most language like, trimming,
8382splitting, and transforming letter case. The < em > string</ em > command
8483also makes it easy to join JSON string arrays into single a string using
0 commit comments