General Computing
linux command-line bash sed
Updated Fri, 20 May 2022 01:27:36 GMT

Bash script: Use variable in sed to delete all but the last "x" lines of a file


I'm using the following bash script to delete all but the last 'x' lines in a log file:

#!/bin/bash
# Script to  write to the log file every minute  and delete all but the last 
# set time in hours
echo `date +\%Y\%m\%d\%H\%M\%S`,`/home/pi/temp/temp` >> /home/pi/temp/temp.log
HOURS="12"                      #Nr of hours in the log
LINES=$(( $HOURS * 60 + 1 ))    #Nr of minutes/lines in the log
echo $LINES                     #Included just for testing if i'm getting the desired amount of lines
sed -e :a -e '$q;N;$LINES,$D;ba' temp.log >temp1.log ;mv temp1.log  temp.log

I substituted the $LINES for its value (12 hours), 721 and the script is working as a cron task, runs every minute, without errors, so i'm sure the error is in the part that includes the variable in the sed.

Also, I just copied this from some website, if someone could be kind enough to explain me what is going on in the sed command, maybe i'd be able to figure out the mistake myself.

This is the error:

sed: -e expression #2, char 8: extra characters after command.

Also, it deletes all the lines in the file.

Solution

You need to take the variables out of single quotes:

sed -e :a -e '$q;N;'"$LINES"',$D;ba'

Other notes:

  • Get out of the habit of using ALL_CAPS variable names. One day you'll write PATH=/tmp and then wonder why your script is broken.
  • % is not a special character for the shell, you don't need to escape it: date +%Y%m%d%H%M%S
  • use $(...) instead of backticks: easier to see and way easier to nest.

  • sed can edit a file in-place, you don't need to write to a temp file and mv it:

    sed -i -e :a -e '$q;N;'"$LINES"',$D;ba'

  • this is very similar to this stackoverflow answer that is well documented. Read that for insight.

Comments (2)

  • +0 – Thanks, will try that ;) as for the caps thing, i really just did that because i saw online on a bash scripting beginners guide, if it wasnt' for that guide i wouldn't have used, but i will change that, thanks :) P.S.: Also, about the editing a file in place, so i just put the filename after the sed command and it will edit it in place? like sed -i -e :a -e '$q;N;'"$LINES"',$D;ba' temp.log ? — Sep 26, 2014 at 10:43  
  • +0 – that's right. . — Sep 26, 2014 at 12:31  

Linked Articles

Local articles referenced by this article:

References