Announcing list2regexp - Generate regular expressions to match lists of strings

198 views
Skip to first unread message

Trevor Slocum

unread,
Sep 17, 2026, 9:16:54 PMSep 17
to golang-nuts
https://codeberg.org/tslocum/list2regexp

Here is an example. Given the following list of strings:

    1.2.3.4
    127.0.0.1
    127.0.0.2
    127.0.0.11
    128.0.0.1
    125.0.0.*
    200.0.0.1
    2001:0db8:85a3:0000:0000:8a2e:0370:1300
    2001:0db8:85a3:0000:0000:8a2e:0370:1330
    2001:0db8:85a3:0000:0000:8a2e:0370:1337
    あいうえお
    あいうえこ
    😃😁😆😐
    😃😁😆
    😃😁🤣

list2regexp will return the following pattern:

    ^(?:1(?:(?:\.(?:2(?:\.(?:3(?:\.(?:4))))))|(?:2(?:(?:5(?:\.(?:0(?:\.(?:0(?:\.(?:.*)))))))|(?:7(?:\.(?:0(?:\.(?:0(?:\.(?:(?:1(?:|(?:1)))|(?:2))))))))|(?:8(?:\.(?:0(?:\.(?:0(?:\.(?:1)))))))))))|(?:2(?:0(?:0(?:(?:\.(?:0(?:\.(?:0(?:\.(?:1))))))|(?:1(?::(?:0(?:d(?:b(?:8(?::(?:8(?:5(?:a(?:3(?::(?:0(?:0(?:0(?:0(?::(?:0(?:0(?:0(?:0(?::(?:8(?:a(?:2(?:e(?::(?:0(?:3(?:7(?:0(?::(?:1(?:3(?:(?:0(?:0))|(?:3(?:(?:0|7))))))))))))))))))))))))))))))))))))))))))|(?:あ(?:い(?:う(?:え(?:(?:お|こ))))))|(?:😃(?:😁(?:(?:😆(?:|(?:😐)))|(?:🤣))))$

Jason E. Aten

unread,
Sep 18, 2026, 1:04:38 AMSep 18
to golang-nuts
Unholy 98 non-capturing groups, Batman!

You might find it interesting to attempt to produce a _minimal_ regular expression to represent the given input set. Its probably a hard problem, but heuristics might get you quite a ways.

In this example, I would undoubtably prefer the list to the regex. That is, I would prefer the simple enumerated list in a switch statement, in vast preference over that eldritch horror of an output regex(!)

Trevor Slocum

unread,
Sep 18, 2026, 5:23:05 AMSep 18
to golang-nuts
Hi Jason,

Thanks for your feedback. Please show how list2regexp does not produce a minimal regular expression to represent the given input set. Using wilcards or CIDR suffixes should produce a minimal pattern when operating on lists of IP addresses and IP address ranges. If you are not operating on lists of IP addresses and ranges, list2regexp shouldn't need any hinting to produce a minimal pattern. Please show a real-world example which would not be better served by using the built-in support for wildcards and CIDR suffixes.

I didn't include a 150,000-line list in my post because that wouldn't be a practical example. If you are only matching a list containing a couple of strings, then yes, using list2regexp probably won't make much of a difference. However, parsing 150,000 IP addresses and CIDR suffixes (or other large lists) with list2regexp will result in a large performance improvement. I just benchmarked the latest version (v1.0.2) and matching against 150,000 addresses and CIDR suffixes takes 25-50 microseconds with warm caches on a 5 year old laptop.

Jason E. Aten

unread,
Sep 18, 2026, 5:40:28 AMSep 18
to golang-nuts
Sorry, I thought I had pointed out what appeared to be excessive in the first line -- the use of non-capturing groups for everything. Maybe that is essential, but its not at all obvious if that is the case. I don't see why they are needed? The regex example would be alot simpler without them.

Jason E. Aten

unread,
Sep 18, 2026, 5:55:58 AMSep 18
to golang-nuts
I do think this is a really neat project. I should have expressed that enthusiasm up front.

The other question I would have before using it: is it guaranteed that only the set of
examples and no others are accepted? I would want to see some fuzz testing: are nearby
strings that should be excluded actually excluded?

Brian Candler

unread,
Sep 18, 2026, 9:45:24 AMSep 18
to golang-nuts
On Thursday, 17 September 2026 at 20:16:54 UTC+1 Trevor Slocum wrote:
list2regexp will return the following pattern:

That's 541 bytes. But if I write this (by hand, untested):

^(?:1\.2\.3\.4|127\.0\.0\.1|127\.0\.0\.2|127\.0\.0\.11|128\.0\.0\.1|125\.0\.0\.\*|200\.0\.0\.1|2001:0db8:85a3:0000:0000:8a2e:0370:1300|2001:0db8:85a3:0000:0000:8a2e:0370:1330|2001:0db8:85a3:0000:0000:8a2e:0370:1337|あいうえお|あいうえこ|😃😁😆😐|😃😁😆|😃😁🤣)$

then it's only 291 bytes - and is much clearer.  Furthermore, I suspect this version runs as fast or faster under go's regexp package, because it builds a DFA anyway.

Jan Mercl

unread,
Sep 18, 2026, 12:25:37 PMSep 18
to Trevor Slocum, golang-nuts
On Thu, Sep 17, 2026 at 9:17 PM Trevor Slocum <tsl...@gmail.com> wrote:
>
> https://codeberg.org/tslocum/list2regexp

I wonder how it compares performance-wise:

jnml@e5-1650:~/tmp$ go install modernc.org/rec@latest
jnml@e5-1650:~/tmp$ rec -lexstringutf8=match -pkg main -import unicode/utf8 1.2.3.4 \

    127.0.0.1 \
    127.0.0.2 \
    127.0.0.11 \
    128.0.0.1 \
    125.0.0.* \
    200.0.0.1 \
    2001:0db8:85a3:0000:0000:8a2e:0370:1300 \
    2001:0db8:85a3:0000:0000:8a2e:0370:1330 \
    2001:0db8:85a3:0000:0000:8a2e:0370:1337 \
    あいうえお\
    あいうえこ\
    😃😁😆😐 \
    😃😁😆 \
    😃😁🤣 > match.go
jnml@e5-1650:~/tmp$



Reply all
Reply to author
Forward
0 new messages