I just have wrote about Extract part of FreeRadius” log with a little awk script. Then I decided that it whould be easier and quicker than with Python.
Here is a Python script, which does the same (and written in the same way):
#!/usr/bin/python
#
#
import sys, re
pattern = sys.argv[1]
file = open(sys.argv[2])
cp = re.compile(pattern)
total = 0
selected = 0
good = False
lines = 0
set = []
while True:
line = file.readline()
if not line:
break
lines += 1
set.append(line)
if cp.search(line):
good = True
if line == '\n':
if good:
print ''.join(set),
selected += 1
good = False
set = []
total += 1
sys.stderr.write("%i records (%i lines) processed\n" %(total, lines))
sys.stderr.write("%i records matched\n" % selected)
sys.stderr.write("Pattern was: '%s'\n\n" % pattern)Nothing special, you see.
What I considered interesting in this script? — it works near 30% faster than awk. And I don’t know how to optimize my awk script :-)
Take a look, this is awk:
time awk -f cutlog.awk pattern='Station-Id = \"XXXYYZ[0-2]\"' detail-YYYYMMDD > detail-YYYYMMDD.part
276358 records (6874776 lines) processed
49574 records matched
Pattern was: 'Station-Id = "XXXYYZ[0-2]"'
33.90user 0.29system 0:34.19elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+306minor)pagefaults 0swaps
This is Python:
time python cutlog.py 'Station-Id = "XXXYYZ[0-2]"' detail-YYYYMMDD > detail-YYYYMMDD.part
276358 records (6874776 lines) processed
49574 records matched
Pattern was: 'Station-Id = "XXXYYZ[0-2]"'
26.60user 0.24system 0:26.85elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+732minor)pagefaults 0swaps
update:
Well, I have tried to «optimize»:
# in place of:
if cp.search(line):
good = True
# to write:
if not good and cp.search(line):
good = True
continue
# and the same for awk script.There is no significant difference. And no significant difference when to change order of checking (first «if line is empty» and then «if line matches» or otherwise), use print instead of printf etc.
English
Українська