Squealer scans a git repository or filesystem for secrets that are being leaked deep within the commit history.
The built-in configuration includes the following checks;
- AnsibleVault
- AWS Manager ID
- AWS MWS key
- AWS Secret Key
- Base64 Encoded Certificates
- DomainPassword
- DSA
- Dynatrace token
- Facebook Creds
- GitCredential
- Github
- Google API key
- Heroku API key
- Keychain file
- KeyStoreFile
- LinkedIn Creds
- MailChimp API key
- Mailgun API key
- NPM Token
- OAuth Token
- OpenAI Secret Key
- OPENSSH
- OpenVPN
- Password literal text
- PayPal Braintree access token
- PGP
- Picatic API key
- Postgres password
- PublishSettings
- RSA
- Shopify credentials
- Slack credentials
- SQL Connection Strings
- Square credentials
- Stripe API key
- Twilio API key
- Twitter credentials
Sometimes we have secrets committed to our projects, generally, we can invalidate them and move on. If Squealer is telling tales about a secret that you are aware of and has been mitigated, you can use the exception rule found in the output to register it as ignored.
curl -s "https://raw.githubusercontent.com/owenrumney/squealer/main/scripts/install.sh" | bashSquealer is intended to be run either locally or as part of a CI process.
./squealer --help
Telling tales on your secret leaking
Usage:
squealer [flags]
Flags:
--concise Reduced output.
--config-file string Path to the config file with the rules.
--debug Include debug output.
--everything Scan all commits.... everywhere.
--from-hash string The hash to work back to from the starting hash.
-h, --help help for squealer
--no-git Scan as a directory rather than a git history.
--output-format string The format that the output should come in (default, json, sarif.
--redacted Display the results redacted.
--to-hash string The most recent hash to start with.squealer git@github.com:owenrumney/woopsie.gitsquealer --no-git /path/to/reporules:
- rule: (A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}
description: Check for AWS Access Key Id
- rule: (?i)aws(.{0,20})?(?-i)['\"][0-9a-zA-Z\/+]{40}['\"]
description: Check for AWS Secret Access Key
- rule: (?i)github[_\-\.]?token[\s:,="\]']+?(?-i)[0-9a-zA-Z]{35,40}
description: Check for Github Token
- rule: https://hooks.slack.com/services/T[a-zA-Z0-9_]{8}/B[a-zA-Z0-9_]{8}/[a-zA-Z0-9_]{24}
description: Check for Slack webhook
- rule: xox[baprs]-([0-9a-zA-Z]{10,48})?
description: Check for Slack token
- rule: "-----BEGIN ((EC|PGP|DSA|RSA|OPENSSH) )?PRIVATE KEY( BLOCK)?-----"
description: Check for Private Asymetric Key
ignore_paths:
- vendor
- node_modules
ignore_extensions:
- .zip
- .png
- .jpg
- .pdf
- .xls
- .doc
- .docx
exceptions:
- exception: release/update.go:D2IDetI6aidl58GE6dv5uAaWmXM=
reason: This is a webhook that we got rid of - can be ignored in this fileThe config file is made up of the rules, ignore_prefixes, ignore_extensions and exceptions.
Rules define the regular expression that is used to detect the secret. Requires a description for posterity.
Ignore paths are folders that you don't want to look ing - generally vendor and the like.
Ignore extensions that have these file types that won't be scanned. Binaries are automatically ignored.
Exceptions are the entries that you've already handled and don't want to be reported any more.
INFO[0000] Using a git scanner to process ../../tfsec/tfsec
INFO[0000] starting at hash 3bd04e7e17f2aad9e5f38826d88325798534a289
Content: | access_key = "AKIAABCD12ABCDEF1ABC"
Filename: | internal/app/tfsec/checks/aws044.go
Line No: | 21
Secret Hash: | bcE9jU2WV11OYs63eGHPZf1l9v8=
Commit: | 4e68e1c5b3bc66982e4b7e6c5cc1c1642c87f83d
Committer: | GitHub (noreply@github.com)
Committed: | 2020-10-21 21:59:22 +0100 +0100
Exclude rule: | internal/app/tfsec/checks/aws044.go:bcE9jU2WV11OYs63eGHPZf1l9v8=
Content: | access_key = "AKIAABCD12ABCDEF1ABC"
Filename: | docs-website/docs/aws/AWS044.md
Line No: | 26
Secret Hash: | bcE9jU2WV11OYs63eGHPZf1l9v8=
Commit: | 8a7715f2cf5a2ac74a1e186792c476fd52ee1474
Committer: | ¨Owen Rumney (owen.rumney@form3.tech)
Committed: | 2021-01-24 19:04:27 +0000 +0000
Exclude rule: | docs-website/docs/aws/AWS044.md:bcE9jU2WV11OYs63eGHPZf1l9v8=
Processing:
duration: 2.99s
commits: 503
commit files: 4095
transgressionMap:
identified: 6
ignored: 0
reported: 2
INFO[0002] Exit code: 1
It's worth noting that these are known because they're examples in the documentation for tfsec - I can add them to the config.yaml as exclusions y using the Exclude rule
Squealer can be used for scanning a specific string using either the default config or by passing your own file in.
go get -u github.com/owenrumney/squealer
package main
import (
"fmt"
"github.com/owenrumney/squealer/pkg/squealer"
"github.com/owenrumney/squealer/pkg/config"
)
func main() {
// create a new scanner (optionally load your own config in)
scanner, err := squealer.New(
squealer.OptionWithConfig(config.DefaultConfig()), // if not supplied , config.DefaultConfig() used
squealer.OptionRedactedSecrets(true), // defaults to true, secrets in output redacted
squealer.OptionNoGitScan(false), // Treat Directories with .git in them as Directories, defaults to false
squealer.OptionWithBasePath("."), // The path to scan, default is '.'
squealer.OptionWithFromHash(""), // Specify the starting hash for the scan, useful for PRs
squealer.OptionWithToHash(""), // Specify the hash to stop scanning, useful for PRs scanning
squealer.OptionWithScanEverything(false), // Scan everything in every branch, defaults to only the current branch
squealer.OptionWithCommitListFile(""), // a file of commits that you want to explicitly scan in a text file.
)
transgressions, err := scanner.Scan()
if err != nil {
panic(err)
}
for _, t := range transgressions {
fmt.Printf("%s[%d]\n", t.Filename, t.LineNo)
}
}
package main
import (
"fmt"
"github.com/owenrumney/squealer/pkg/squealer"
)
func main() {
// create a new scanner (optionally load your own config in)
scanner := squealer.NewStringScanner()
testString := `password = "superSecretPassword"`
// scan the string and if a transgression is found, report it.
if result := scanner.Scan(testString); result.TransgressionFound {
fmt.Printf("found an issue in [%s]. %s\n", testString, result.Description)
}
}Image by George Rumney