fjordtools / Errors / Regex catastrophic backtracking

Regex catastrophic backtracking (ReDoS)

Catastrophic backtracking happens when a pattern can split the same text many ways and the match then fails. The engine tries every split. A repeat inside a repeat, such as (a+)+$, doubles the work with each extra character. Remove the overlap, make the inner part unambiguous, or use an engine with guaranteed linear time.

Fjordtools team · Seen in Backtracking engines (JavaScript, Python, Java, PCRE) · Published · 1 min read

How browsers word it

fjordtools regex testerThis pattern took longer than 1.5 seconds. It may backtrack catastrophically.
Browsers and NodeNo error: the tab or process stops responding until the match ends
Python and JavaNo error: one thread runs at full CPU until the match ends

Common causes

  1. A repeat inside a repeat

    (a+)+$ on 26 letters a followed by b takes about two seconds in V8; each extra letter roughly doubles it. The pattern fails at the b, then tries every way to split the a's between the two repeats.

  2. Alternatives that overlap

    ^(a|aa)+$ can read the same run of a's as many different mixes of a and aa. When the end does not match, all of them are tried.

  3. Two open-ended parts next to each other

    Patterns like (.*)*, or .*.*x on a line without an x, give the engine a choice at every position. Only the failing case is slow, so tests on good input pass.

  4. The unambiguous form is fast

    a+$ means the same here and has one way to read the text. It fails on a long run of a's in a fraction of a millisecond.

How to fix it

  1. Remove the inner repeat: (a+)+ matches the same text as a+.
  2. Make alternatives start with different characters, or merge them: (a|aa)+ is a+.
  3. Anchor the pattern and use a character class that cannot match the separator, such as [^,]* instead of .*.
  4. On user input, set a time limit or use a linear-time engine such as RE2.
  5. Test with a long non-matching input: the tester stops after 1.5 seconds and tells you.

Related errors

More error explainers are coming

See all errors.