Unix & Linux
perl
Updated Mon, 19 Sep 2022 03:21:57 GMT

Perl: speed of s/// with variable in replacement


This:

while($read=sysread(STDIN,$buf,32760)) {
    $buf=~s/\r/posttag\rpretag\t/go;
    $buf=~s/\n/posttag\npretag\t/go;
    syswrite(STDOUT,$buf);
}

delivers ~200 MB/s on my system.

This:

my $pretag = "pretag";
my $posttag = "posttag";
while($read=sysread(STDIN,$buf,32760)) {
    $buf=~s/\r/$posttag\r$pretag\t/go;
    $buf=~s/\n/$posttag\n$pretag\t/go;
    syswrite(STDOUT,$buf);
}

delivers ~100 MB/s on my system.

Why?

I thought that when I had used /o it should not matter if the content is a variable or a fixed string. Is there an easy way I can get the speed of the first?




Solution

Are you sure that regexp recompilation is to blame? You can get information about what perl is doing with your regexps using

use re 'debug';

In second case you introduce interpolated string in s//HERE/ and perl interpolates it with each iteration.

Try to rewrite like

my $pretag = 'pretag';
my $posttag = 'posttag';
my $first_replace = "$posttag\r$pretag\t";
my $second_replace = "$posttag\n$pretag\t";
while($read=sysread(STDIN,$buf,32760)) {
    $buf=~s/\r/$first_replace/go;
    $buf=~s/\n/$second_replace/go;
    syswrite(STDOUT,$buf);
}




Comments (3)

  • +0 – How can I tell Perl not to interpolate it every time? It seems each interpolation slow it down. — Jul 31, 2022 at 09:38  
  • +0 – doesn't the above example help? — Jul 31, 2022 at 10:15  
  • +0 – Here , $pretag & $posttag will be interpolated once into $first_replace & into $second_replace but these will be interpolated every time into the regex ! Question Ex1 contains no interpolation (fast) & Question Ex2 contains interpolation (slow) ( Both given by OP @OleTange ) & Answer Ex contains interpolation (slow) !! — Aug 01, 2022 at 07:58