Brownian motion

I’ve decided to make it available after some nice people asked for this.

Now i have no time and, let’s say, health to make it better, so please don’t complain :-)

I guess it maybe would be better to make it available somewhere at code.google.com — i would like to work on this code later, surely.

Anyway. Now — as it now is. With incomplete README:

Hi.

Sorry, this is a mess.

Look at settings.py, of course. And there is one more file,
shap_defaults.py, take a look, make changes.

This configurator assumes that you have a linux box with two network
interfaces connected in a bridge (two-ports Ethernet switch), and you
wish to set up a HTB shaper on each; and these shapers should be
symmetrical, one for incoming (destined to your IP(s)) and another for
outgoing (from your IP(s)) traffic.

This configurator assumes that you already do have a script, which
should set up your shaper's "basement" --- root qdisc and root
class(es). With their leaves (qdiscs and classes) possibly. With ---
important --- hash tables for all your /24s.

An i assume you can set up django and add this "project" to it .)

Then you can set up (via django admin interface or using fixtures) your
networks, all one by one, as /24 prefixes (2 for /23, 4 for /22 etc..).

You can (and should, actually) configure:
  -  "QoS classes" (in terms of "which class should be a parent for this
     client's class" and "which part of ceil should be guaranteed").
  -  contract types (to use identifiers for contracts in a form
     "<letter_or_some_letters>-<digits><optional_prefix>")
  -  ?.. something else?.. Well, i will update this README, i hope.

You may configure:
  -  "own" client's networks (which, for example, your client announces
     to you via some of dynamic routing protocols, or which are set up
     statically by you own)
  -  network ranges delegations --- ugly, "incomplete" (if smth else may
     be considered as complete)

Now --- i am really sorry, *really* :-) --- there is no gettext support,
and there are a lot of comments and remarks in Ukrainian.

Now there are a lot (no too much though) places with unused code or like
this --- *this is a mess*.

I am not a programmer, at all. I wish _you_ could help me to do this
better.)

"AS IS", surely, and this README --- "AS IS".

Sorry.

# .-)
# vim: set tw=72:

I am not even sure this can be deployed and used. It works for me though, works great. So i hope i will update it to some more usable state.

Please see some info here. I will translate this post into English, i hope.

;;
 
(use-modules (ice-9 rdelim) (ice-9 regex))
 
(define argv (program-arguments))
 
(define filename (car (cdr (cdr argv))))
(define inputfile (open-input-file filename))
 
(define pattern-string (cadr argv))
(define pattern (make-regexp pattern-string))
 
 
(define records 0)
(define lines 0)
(define selected 0)
 
(define (read-all-lines)
    (let loop ((stack '()) (good #f))
        (let ((line (read-line inputfile)))
            (set! stack (append stack (list line)))
            (set! lines (+ 1 lines))
 
            (if (regexp-exec pattern line)
                (set! good #t)
                (if (equal? "" line)
                    (begin
                        (if (eq? good #t)
                            (begin
                                (for-each (lambda (line) (display line) (newline)) stack)
                                (set! selected (+ 1 selected))))
                        (set! good #f)
                        (set! stack '())
                        (set! records (+ 1 records)))))
 
            (if (not (eof-object? (peek-char inputfile)))
                (loop stack good)))))
 
(read-all-lines)
 
(use-modules (ice-9 format))
 
(format (current-error-port)
"~d records (~d lines) processed
~d records matched
Pattern was: '~a'
" records lines selected pattern-string)
 
;; vim: ts=2:

Scheme:

$ time guile -s fradlog_extract.scm 'Station-Id = \"4494.....\"' detail-20090519 > part-scheme
183764 records (4563405 lines) processed
447 records matched
Pattern was: 'Station-Id = "4494....."'
 
real    0m27.653s
user    0m27.550s
sys     0m0.110s

awk:

$ time awk -f fradlog_extract.awk pattern='Station-Id = \"4494.....\"' detail-20090519 > part-awk
183764 records (4563405 lines) processed
447 records matched
Pattern was: 'Station-Id = "4494....."'
 
real    0m21.680s
user    0m21.490s
sys     0m0.090s

python:

$ time python fradlog_extract.py 'Station-Id = "4494....."' detail-20090519 > part-python
183764 records (4563405 lines) processed
447 records matched
Pattern was: 'Station-Id = "4494....."'
 
real    0m9.766s
user    0m9.670s
sys     0m0.060s

>>> (with-error-to-file (current-error-port) (display "foo"))
  File "", line 1
    (with-error-to-file (current-error-port) (display "foo"))
        ^
SyntaxError: invalid syntax
>>>

Trying to learn Scheme, trying to work with Python…

Yes, right, guile throws another error message, i’ve already learned that :O)

Well.. I’ve done my django based configurator for my HTB shaper (bridge/linux).

I will try to translate this post in english; if you can read Ukrainian, try here.

I just have wrote about Extract part of FreeRadius” log with a little awk script. Then I decided that it whould be easier and quicker than with Python.

Here is a Python script, which does the same (and written in the same way):

#!/usr/bin/python
#
#
 
import sys, re
 
pattern = sys.argv[1]
file = open(sys.argv[2])
 
cp = re.compile(pattern)
 
total = 0
selected = 0
good = False
lines = 0
 
set = []
 
while True:
    line = file.readline()
    if not line:
        break
 
    lines += 1
 
    set.append(line)
 
    if cp.search(line):
        good = True
 
    if line == '\n':
        if good:
            print ''.join(set),
            selected += 1
 
        good = False
        set = []
        total += 1
 
sys.stderr.write("%i records (%i lines) processed\n" %(total, lines))
sys.stderr.write("%i records matched\n" % selected)
sys.stderr.write("Pattern was: '%s'\n\n" % pattern)

Nothing special, you see.

What I considered interesting in this script? — it works near 30% faster than awk. And I don’t know how to optimize my awk script :-)

Take a look, this is awk:

time awk -f cutlog.awk pattern='Station-Id = \"XXXYYZ[0-2]\"' detail-YYYYMMDD > detail-YYYYMMDD.part
276358 records (6874776 lines) processed
49574 records matched
Pattern was: 'Station-Id = "XXXYYZ[0-2]"'
 
33.90user 0.29system 0:34.19elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+306minor)pagefaults 0swaps

This is Python:

time python cutlog.py 'Station-Id = "XXXYYZ[0-2]"' detail-YYYYMMDD > detail-YYYYMMDD.part
276358 records (6874776 lines) processed
49574 records matched
Pattern was: 'Station-Id = "XXXYYZ[0-2]"'
 
26.60user 0.24system 0:26.85elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+732minor)pagefaults 0swaps

update:
Well, I have tried to «optimize»:

    # in place of:
    if cp.search(line):
        good = True
    # to write:
    if not good and cp.search(line):
        good = True
        continue
# and the same for awk script.

There is no significant difference. And no significant difference when to change order of checking (first «if line is empty» and then «if line matches» or otherwise), use print instead of printf etc.