8000 GitHub - lizmat/App-Rak: 21st century grep / find / ack / ag / rg on steroids · GitHub
[go: up one dir, main page]
Skip to content
lizmat
App-Rak
Public

About

21st century grep / find / ack / ag / rg on steroids

raku.land/zef:lizmat/App::Rak

Resources

Readme
Artistic-2.0 license
Activity

Stars

160 stars

Watchers

3 watching

Forks

7 forks
Report repository
BranchesTags

Latest commit

 

History

308 Commits

Folders and files

Repository files navigation

Actions Status Actions Status Actions Status

NAME

App::Rak - 21st century grep / find / ack / ag / rg on steroids

SYNOPSIS

# look for "foo" in current directory recursively
$ rak foo

# look for "foo" in directory "bar" recursively
$ rak foo bar

# look for "foo" as word in current directory
$ rak '/ << foo >> /'

# look for "foo", only produce filenames
$ rak foo --files-with-matches

# also produce 2 lines before and after
$ rak foo --before=2 --after=2

# lines with foo AND bar
$ rak foo --and=bar

DESCRIPTION

App::Rak provides a CLI called rak that allows you to look for a pattern in (a selection of files) from one or more directories recursively. It has been modelled after utilities such as grep, ack, ag and rg, with a little bit of find mixed in, and -n and -p parameters of many programming languages.

Note: this project is now in beta-development phase. Comments, suggestions and bug reports continue to be more than welcome!

PATTERN SPECIFICATION

Patterns will be interpreted in the following ways if no --type has been specified, or --type=auto has been specified.

/ regex /

If the pattern starts and ends with /, then it indicates a Raku regex. No special processing of the given string between slashes will be done: the given pattern will be parsed as a regex verbatim. During the search process, each item will be matched against this regex. Any --ignorecase or --ignoremark arguments will be honoured.

One can also specify the regex type with --type=regex, so that / foo / can be considered a shortcut for --type=regex foo.

{ code }

If the pattern starts with { and ends with }, then it indicates Raku code to be executed. No special processing of the given string between the curly braces will be done: the given code will be compiled as Raku code. During the search process, this code will be run for each item, available in $_. To facilitate the use of libraries that wish to access that topic, it is also available as the $*_ dynamic variable.

The dynamic variable $*SOURCE will contain the IO::Path object of the file being processed. Note that the Raku code will be called in a thread unsafe manner.

The dynamic variable $*_ will contain the topic with which the code was called. This to allow custom libraries to easily obtain the topic without the user needing to specify that again.

One can also specify the code type with --type=code, so that { .uc } can be considered a shortcut for --type=code .uc.

*code

If the pattern starts with *, then this is a short way of specifying Raku code as a pattern, using Whatever-currying semantics. Otherwise the same as { code }.

^string

If the pattern starts with ^, then it indicates the string should be at the start of each item. Basically a shortcut to specifying --type=starts-with string. Any --smartcase, --smartmark, --ignorecase or --ignoremark arguments will be honoured.

string$

If the pattern ends with $, then it indicates the string should be at the end of each item. Basically a shortcut to specifying --type=ends-with string. Any --smartcase, --smartmark, --ignorecase or --ignoremark arguments will be honoured.

^string$

If the pattern starts with ^ and ends with $, then it indicates that the string should be equal to the item. Basically a shortcut to specifying --type=equal string. Any --smartcase, --ignorecase or --ignoremark arguments will be honoured.

§string

If the pattern starts with §, then it indicates that the string should occur as a word (with word-boundaries on both ends) in the item. Basically a shortcut to specifying --type=words string. Any --smartcase, --smartmark, --ignorecase or --ignoremark arguments will be honoured.

!specification

If the pattern starts with !, then it indicates that the result of the pattern specification should be negated.

&specification

If the pattern starts with &, then it indicates that the result of the pattern specification should be true as well as the result of the previous pattern specification.

s:string

If the pattern starts with 's:', then it indicates that the string should be split on whitespace, and each part should be considered a separate pattern specification. Basically a shortcut to specifying --type=split string.

jp:path

If the pattern start with 'jp:', then interpret the rest of the pattern as a JSON path. Only makes sense when used together with --json-per-file, --json-per-line or --json-per-elem. Requires that the JSON::Path module is installed. Basically a shortcut to specifying path --type=json-path.

string

If there are no special start or end markers, then it indicates that the string should occur somewhere in the item. Basically a shortcut to specifying string --type=contains. Any --smartcase, --smartmark, --ignorecase or --ignoremark arguments will be honoured.

POSITIONAL ARGUMENTS

pattern

The pattern to search for, if no pattern was specified any other way.

Patterns can also be specified with the --pattern, --not and --patterns-from options, in which case all other positional arguments are considered to be path specifications.

path(s)

Optional. Either indicates the path of the directory (and its sub-directories), or the file that will be searched, or a URL that will produce a file to be searched. By default, all directories that do not start with a period, and which are not symbolic links, will be recursed into (but this can be changed with the --dir option).

By default, all files with known extensions will be searched in the directories. This can be changed with the --file option, or specialized version of that like --extensions.

Paths can also be specified with the --paths option, in which case there should only be a positional argument for the pattern, or none if --pattern option was used for the pattern specification.

ON CALLABLES AS PATTERN

Callables can be specified by a string starting with *. (so-called Whatever currying, or as a string starting with { and ending with }.

Note that if a Callable is specified as a pattern, then no highlighting can be performed as it cannot signal why or where a match occurred.

The return value of the pattern Callable match is interpreted in the following way:

True

If the Boolean True value is returned, assume the pattern is found. Produce the item unless --invert-match was specified.

False

If the Boolean False value is returned, assume the pattern is not found. Do not produce the item unless --invert-match was specified.

Nil

If Nil is returned, assume the pattern is not found.

This typically happens when a try is used in a pattern, and an execution error occurred. Do not produce the item unless --invert-match was specified.

Empty

If the empty Slip is returned, assume the pattern is not found. Do not produce the item unless --invert-match was specified. Shown in stats as a passthru.

any other Slip

If a non-empty Slip is returned, produce the values of the Slip separately for the given item (each with the same item number).

any other value

Produce that value.

PHASERS IN CALLABLE PATTERNS

The Raku Programming Language has a number of unique features that can be used with patterns that are executable code. One of them is the use of so-called phasers (pieces of code that will be executed automatically when a certain condition has been met.

App::Rak currently supports all of Raku's loop phasers:

phaser event
FIRST code to run when searching starts
NEXT code to run when searching a file is done
LAST code to run when searching is done

These phasers will be called in a thread-safe manner.

# show number of files inspected before the search result
$ rak '{ state $s = 0; NEXT $s++; LAST say "$s files"; .contains("foo")}'

# show number of files inspected after of the search result
$ rak '{ state $s = 0; NEXT $s++; END say "$s files"; .contains("foo")}'

Note that the use of the LAST phaser will make the search run eagerly, meaning that no results will be shown until the search has been completed.

Any other phasers that do not require special attention by App::Rak are also supported in any code specified (such as BEGIN and END).

ON THE INTERPRETATION OF OPTIONS

All options when using App::Rak, start with either one dash - or two dashes --.

If an option starts with two dashes, it is a so-called "long option". Any characters after the dashes are considered to be the single name of the option.

If an option sta 10BC8 rts with a single dash, then it is considered to be a collection of "short options", each of 1 letter. If the number of short options is 1, then it can be followed by a numerical value (without equal sign).

If the specification of the option does not contain an equal sign =, then the option is interpreted as a boolean option. By default, such a flag is considered to represent True. The value can be negated in two ways:

  • a slash before the name

  • the string "no-" before the name

Some examples:

  • -i

Option "i" is True.

  • -j5

Option "j" has the value 5.

  • -im

Options "i" and "m" are True.

  • -/i

Option "i" is False.

  • -/im

Options "i" and "m" are False.

  • -no-i

Option "i" is False.

  • --foo

Option "foo" is True.

  • --/foo

Option "foo" is False.

  • --no-foo

Option "foo" is False.

If the specification of an option contains an equal sign after the name, then whatever follows that, is considered the value of that option. Whether or not that value needs to be quoted, and how they are to be quoted, really depends on the shell that you use to access rak. Generally, if the value consists of alphanumeric characters only, no quoting is necessary. Otherwise it's better to quote your values.

Some examples:

  • -s=foo

Option "s" has the value "foo".

  • -t='foo bar'

Option "t" has the value "foo bar".

  • --frobnicate=yes

Option "frobnicate" has the value "yes".

CREATING YOUR OWN OPTIONS

App::Rak provides many options. If you are happy with a set of options for a certain workflow, You can use the --save option to save that set of options and than later access them with the given name:

# create -i shortcut for ignoring case
$ rak --ignorecase --save=i
Saved option '-i' as: --ignorecase

# create -m shortcut for ignoring accents
$ rak --ignoremark --save=m
Saved option '-m' as: --ignoremark

# same as --ignorecase --ignoremark
$ rak foo -im

Generally speaking, the most used boolean options can be saved as single letter options: this allows multiple options to be specified in a single, short manner (as shown above).

To better document / remember what a particular custom option is meant to do, you can add a description with the --description option.

# add a description to the -i custom option
$ rak --description='Search without caring for uppercase' --save=i
Saved '--description='Search without caring for uppercase'' as: -i

# add an option -g for searching only git files
$ rak --description='Committed files only' --under-version-control --save=g
Saved '--description='Committed files only' --under-version-control' as: -g

There is also a special named option that indicates the options that will be automatically activated on any invocation: (default).

# enable --smartcase by default on every run
$ rak --smartcase --save='(default)'
Saved '--smartcase' as: (default)

You can use the --list-custom-options to see what options you have saved before.

Custom options are saved in ~/.rak-config.json. You can override this by specifying the RAK_CONFIG environment variable.

# read custom options from ".custom.json" (in the current directory)
$ RAK_CONFIG=.custom.json rak foo

You can also use the RAK_CONFIG variable to disable loading any configuration by not specifying a value:

# start rak without any custom configuration
$ RAK_CONFIG= rak foo

SUPPORTED OPTIONS

All options are optional. Any unexpected options, will cause an exception to be thrown with the unexpected options listed and possible alternatives mentioned. Unless specifically indicated otherwise, using the negation of a flag has the same effect as not specifying it.

--absolute

Flag. If specified indicates that whenever paths are shown, they will be shown as absolute paths. Defaults to False, which will cause paths to be produced as paths relative to the current directory.

--accept=code

Specifies the code that should be executed that should return True if the path is acceptable, given an IO::Path object of the path. See also --deny.

# Include files that have "use Test" in them
$ rak --accept='*.slurp.contains("use Test")'

--accessed=condition

If specified, indicates the Callable that should return True to include a file in the selection of files to be checked. The access time of the file (number of seconds since epoch, as a Num value) will be passed as the only argument. Note that many file systems do not actually support this reliably.

See "CHECKING TIMES ON FILES" for more information about features that can be used inside the Callable.

--ack

Attempt to interpret following arguments as if they were arguments to the ack utility. This is incomplete and may be subtly different in behaviour. Intended as a temporary measure for people used to using ack, while they train their muscle memory to use rak instead.

If you'd like a list of the option configuration, you can use the --list-custom-options argument.

# Show the current mappinng of "ack" to "rak" options
$ rak --ack --list-custom-options

--after-context=N

Indicate the number of lines that should be shown after any line that matches. Defaults to 0. Will be overridden by a --context argument.

--allow-loose-escapes

Only applicable if --csv-per-line has been specified. Flag. If specified, indicates that any character may be escaped.

--allow-loose-quotes

Only applicable if --csv-per-line has been specified. Flag. If specified, indicates that fields do not need to be quoted to be acceptable.

--allow-whitespace

Only applicable if --csv-per-line has been specified. Flag. If specified, indicates that whitespace is allowed around separators.

--also-first[=N]

Indicates the number of lines from the start of the file that should be shown if there is a match. If not specified, 0 will be assumed. If specified as a flag, 1 will be assumed.

--always-first[=N]

Indicates the number of lines from the start of the file that will always be shown, regardless of whether there is a match or not. If not specified, 0 will be assumed. If specified as a flag, 1 will be assumed.

--auto-decompress

Flag. If specified with a True value, will accept compressed files with the .gz (gzip) or .bz2 (bzip2) extension, if the extension was otherwise acceptable. Will automatically decompress files for inspection.

--and=foo

Specify additional pattern that should match as well as any previously specified pattern to produce a full match.

--andnot=foo

Specify additional pattern that should not match as well as any previously specified pattern to produce a full match.

--auto-diag

Only applicable if --csv-per-line has been specified. Flag. If (implicitly) specified, will show diagnostic information about problems that occurred during parsing of the CSV file. The default is True.

--backtrace

Flag. When specified with a True value, will interpret either standard input, or a single file, as a Raku backtrace. And produce a result containing the lines of source code from that backtrace. Can be used together with --context, --before-context, --after-context, --edit and --vimgrep. Any pattern specification will only be used for highlighting. If not used in combination with --edit or --vimgrep, will assume a context of 2 lines.

# look at source of a stacktrace
$ raku script 2>&1 | rak --backtrace

# inspect the source of a stacktrace in an editor
$ raku script 2>&1 | rak --backtrace --edit

# inspect a backtrace stored in a file
$ rak --backtrace filename

--backup[=extension]

Indicate whether backups should be made of files that are being modified. If specified without extension, the extension .bak will be used.

--batch[=N]

Indicate the number of files that should be checked per thread. If specified as a flag, will assue 1. Defaults to 64 if not specified. See also <--degree>.

--before-context=N

Indicate the number of lines that should be shown before any line that matches. Defaults to 0. Will be overridden by a --context argument.

--blame-per-file

Flag. Only makes sense if the pattern is a Callable. If specified, indicates that each of the selected files will be provided as Git::Blame::File objects if git blame can be performed on the a selected file. If that is not possible, then the selected file will be ignored.

If information can be obtained, then the associated Git::Blame::File object will be presented to the pattern Callable. If the Callable returns True, then the filename will be produced. If anything else is returned, then the stringification of that object will be produced.

Requires the Git::Blame::File Raku module to be installed.

# show files with more than 10 commits
$ rak '*.commits > 10' --blame-per-file --files-with-matches

--blame-per-line

Flag. Only makes sense if the pattern is a Callable. If specified, indicates that each line from the selected files will be provided as Git::Blame::Line objects if git blame can be performed on the a selected file. If that is not possible, then the selected file will be ignored.

If information can be obtained, then the associated Git::Blame::Line object will be presented to the pattern Callable. If the Callable returns True, then the short representation of the git blame information will be produced. If the returned value is anything else, then the stringification of that object will be produced.

# show git blame on lines of which the author is "Scooby Doo"
$ rak '{ .author eq "Scooby Doo" }' --blame-per-line

Requires that the Git::Blame::File module is installed.

--blocks=condition

If specified, indicates the Callable that should return True to include a file in the selection of files to be checked. The number of logical blocks that a file takes up in the filesystem, will be passed as the only argument.

# show files that consist of at least 3 blocks
$ rak --find --blocks='* >= 3'

--break[=string]

Indicate whether there should be a visible division between matches of different files. Can also be specified as a string to be used as the divider. Defaults to True (using an empty line as a divider) if --group-matches is (implicitly) set to True, else defaults to False.

--checkout=branch

Only valid if the current directory is under git version control. Indicate the branch to checkout by the general matching logic of App::Rak. Will produce listing of matching branches if more than one, or say that there is no match. Branches need not have been checked out locally yet.

--categorize=categorizer

If specified, indicates the Callable that should return zero or more keys for a given line to have it categorized. This effectively replaces the filename if a line by its key in the result. See also --classify.

# categorize by the first two letters of a line
$ rak --categorize='*.substr(0,2).comb'

--classify=classifier

If specified, indicates the Callable that should return a key for a given line to have it classified. This effectively replaces the filename if a line by its key in the result. See also --categorize.

# classify by the last letter of a line
$ rak --classify='*.substr(*-1)'

--context=N

Indicate the number of lines that should be produced around any line that matches. Defaults to 0.

--count-only

Flag. Indicate whether just the number of lines with matches should be calculated. When specified with a True value, will show a "N matches in M files" by default, and if the :files-with-matches (or files-without matches) option is also specified with a True value, will just show total counts. See also --stats-only.

--created=condition

If specified, indicates the Callable that should return True to include a file in the selection of files to be checked. The creation time of the file (number of seconds since epoch, as a Num value) will be passed as the only argument.

See "CHECKING TIMES ON FILES" for more information about features that can be used inside the Callable.

--csv-per-line

Flag. Only makes sense if the pattern is a Callable. If specified with a True value, indicates that selected files should be interpreted as comma separated values (CSV). Each row from the selected files will be provided as a list of strings (or as CSV::Field objects if --keep-meta was specified).

Attempt to interpret file as a CSV file, and pass each row as a List to to the pattern Callable. Only files with extensions from the #csv group will be tried, unless overridden by any explicit extension specification.

More documentation can be found with the Text::CSV module itself.

# Show the values of the column named "foo" of the rows in the "info.csv"
# file if the column named "bar" is equal to "foo"
$ rak --csv-per-line '{.<foo> if .<bar> eq "foo"}' info.csv

# Show the values of the first column of the rows in the "info.csv" file
# if the second column is equal to "foo"
$ rak --csv-per-line --/headers '{.[0] if .[1] eq "foo"}' info.csv

--degree[=N | code]

Indicate the number of worker threads that should be maximally. Defaults to the number of cores minus 1 if not specified. Assumes 1 if specified as a flag. Can also take a Callable specification, in which case the number of CPU cores will be presented to that Callable as the only argument. See also <--batch>.

--deny=code

Specifies the code that should be executed that should return True if the path is NOT acceptable, given an IO::Path object of the path. See also --accept.

# Include files that **NOT** have "use Test" in them
$ rak --deny='*.slurp.contains("use Test")'

--description=text

Specify a description to be saved with the custom option. This will be shown prominently with --list-custom-options. If it is the only argument apart from --save, then the discription will be added (if there was no description yet) or replace the current description of the option.

--device-number=condition

If specified, indicates the Callable that should return True to include a file in the selection of files to be checked. The device number of the filesystem on which the file is located, will be passed as the only argument.

--dir=condition

If specified, indicates the Callable that should return True to have a directory be included for further recursions in file selection. The basename of the directory will be passed as the only argument. Defaults to all directories that do not start with a period. Can specify as a flag to include all directories for recursion.

--dont-catch

Flag. If specified as a flag, will not catch any error during processing, but will throw any error again. Defaults to False, making sure that errors will be caught. Mainly intended for debugging and error reporting.

--dryrun

Flag. Indicate to not actually make any changes to any content modification if specified with a True value. Only makes sense together with the --modify-files and the --rename-files option.

--eco-code=[rea|fez]

Indicate that all the files that can contain Raku code in the local ecosystem cache (as generated by Ecosystem::Cache) should be searched. If specified as a flag, will assume the "rea" (Raku Ecosystem Archive).

--eco-doc=[rea|fez]

Indicate that all the files that can contain documentation in the local ecosystem cache (as generated by Ecosystem::Cache) should be searched. If specified as a flag, will assume the "rea" (Raku Ecosystem Archive).

--eco-meta[=name1,name2]

Intended to be used by Raku ecosystem maintainers. Indicates the name of zero or more Raku ecosystems of which to inspect the META6.json information of all its modules. Currently supported names are:

name description
p6c the original git ecosystem (deprecated)
cpan the ecosystem piggybacking on PAUSE / CPAN (deprecated)
fez the currently recommended ecosystem for new modules / updates
rea the Raku Ecosystem Archive

Defaults to rea if specified as a flag. Implies --json-per-elem.

# show all unique module names by an author from the REA
$ rak '{ .author eq "Scooby Doo" }' --eco-meta

# same, but now from the p6c and cpan ecosystems
$ rak '{ .author eq "Scooby Doo" }' --eco-meta=p6c,cpan

Assumes zef is installed and its meta information is available.

--eco-provides=[rea|fez]

Indicate that the module files (from the "provides" section of the meta-information in a distribution) in the local ecosystem cache (as generated by Ecosystem::Cache) should be searched. If specified as a flag, will assume the "rea" (Raku Ecosystem Archive).

--eco-scripts=[rea|fez]

Indicate that the scripts (from the "bin" directory of a distribution) in the local ecosystem cache (as generated by Ecosystem::Cache) should be searched. If specified as a flag, will assume the "rea" (Raku Ecosystem Archive).

--eco-tests=[rea|fez]