Brownian motion

The quick brown fox jumps over the lazy dog

 

Archive for the Category ‘Programming’

Making RtConfig (IRRToolSet) in Ubuntu

Having used IRRToolSet in elder Ubuntu releases (elder compilers, libraries etc), i finally decided to compile it under 9.10 (yes, previously made RtConfig works under 9.10 too, «but anyway»).

But something goes wrong, with several files — illegal conversion and «bad» syntax and the like.

So, if you’ll run into the same problems, hope this patch will help.

Apply patch as usually (in the directory where IRRToolSet-4.8.5 is unpacked):

$ patch -p0 --dry-run < IRRToolSet-4.8.5-ubu-patch.diff

If everything is OK, remove --dry-run switch and run again.

Django-based HTB shaper configurator

I’ve decided to make it available after some nice people asked for this.

Now i have no time and, let’s say, health to make it better, so please don’t complain :-)

I guess it maybe would be better to make it available somewhere at code.google.com — i would like to work on this code later, surely.

Anyway. Now — as it now is. With incomplete README:

Hi.

Sorry, this is a mess.

Look at settings.py, of course. And there is one more file,
shap_defaults.py, take a look, make changes.

This configurator assumes that you have a linux box with two network
interfaces connected in a bridge (two-ports Ethernet switch), and you
wish to set up a HTB shaper on each; and these shapers should be
symmetrical, one for incoming (destined to your IP(s)) and another for
outgoing (from your IP(s)) traffic.

This configurator assumes that you already do have a script, which
should set up your shaper's "basement" --- root qdisc and root
class(es). With their leaves (qdiscs and classes) possibly. With ---
important --- hash tables for all your /24s.

An i assume you can set up django and add this "project" to it .)

Then you can set up (via django admin interface or using fixtures) your
networks, all one by one, as /24 prefixes (2 for /23, 4 for /22 etc..).

You can (and should, actually) configure:
  -  "QoS classes" (in terms of "which class should be a parent for this
     client's class" and "which part of ceil should be guaranteed").
  -  contract types (to use identifiers for contracts in a form
     "<letter_or_some_letters>-<digits><optional_prefix>")
  -  ?.. something else?.. Well, i will update this README, i hope.

You may configure:
  -  "own" client's networks (which, for example, your client announces
     to you via some of dynamic routing protocols, or which are set up
     statically by you own)
  -  network ranges delegations --- ugly, "incomplete" (if smth else may
     be considered as complete)

Now --- i am really sorry, *really* :-) --- there is no gettext support,
and there are a lot of comments and remarks in Ukrainian.

Now there are a lot (no too much though) places with unused code or like
this --- *this is a mess*.

I am not a programmer, at all. I wish _you_ could help me to do this
better.)

"AS IS", surely, and this README --- "AS IS".

Sorry.

# .-)
# vim: set tw=72:

I am not even sure this can be deployed and used. It works for me though, works great. So i hope i will update it to some more usable state.

Shaper as an effective firewall

Hash-tables based shaper can act as an effective firewall; just take a look.

Now our shaper configurator generates, let’s say, this script for a client’s connection (excerpt):

#
# Contract I-1082,
# connection 606 (0x25e).
#
# input:
/sbin/tc class add dev clients0 classid 1:25e parent 1:fe10 htb rate 96kbit ceil 128kbit quantum 1500 burst 7500 cburst 12500 prio 50
/sbin/tc filter add dev clients0 protocol 802.1q parent 1:0 prio 100 u32 ht 133:5c match ip dst X.X.133.92 flowid 1:25e
/sbin/tc filter add dev clients0 protocol 802.1q parent 1:0 prio 100 u32 ht 133:5d match ip dst X.X.133.93 flowid 1:25e
/sbin/tc filter add dev clients0 protocol 802.1q parent 1:0 prio 100 u32 ht 133:5e match ip dst X.X.133.94 flowid 1:25e
/sbin/tc filter add dev clients0 protocol 802.1q parent 1:0 prio 100 u32 ht 133:5f match ip dst X.X.133.95 flowid 1:25e

Now let’s assume we have a lot of DoS traffic from IP 173.204.53.138 to IP X.X.133.94 — a lot of packets, a client’s bandwidth is exhausted. (Yes, that’s real IP, that was a real DoS attack,-)

All we need is to write a filter into a proper hash table cell:

# input:
/sbin/tc class add dev clients0 classid 1:25e parent 1:fe10 htb rate 96kbit ceil 128kbit quantum 1500 burst 7500 cburst 12500 prio 50
# FIXME:
/sbin/tc filter add dev clients0 protocol 802.1q parent 1:0 prio 100 u32 ht 133:5e match ip src 173.204.53.138 police mtu 1 drop flowid 1:fe49
#
/sbin/tc filter add dev clients0 protocol 802.1q parent 1:0 prio 100 u32 ht 133:5c match ip dst X.X.133.92 flowid 1:25e
/sbin/tc filter add dev clients0 protocol 802.1q parent 1:0 prio 100 u32 ht 133:5d match ip dst X.X.133.93 flowid 1:25e
/sbin/tc filter add dev clients0 protocol 802.1q parent 1:0 prio 100 u32 ht 133:5e match ip dst X.X.133.94 flowid 1:25e
/sbin/tc filter add dev clients0 protocol 802.1q parent 1:0 prio 100 u32 ht 133:5f match ip dst X.X.133.95 flowid 1:25e

«FIXME» — because it’s a manual work, i have to add this feature in my configurator :-)

This filter we have added to a cell, where all filters regarding packets to X.X.133.94 are (there are only one filter normally). And we added it before «normal» filter, which «routes» packets to a client’s class.

This work like this: if a packet from IP 173.204.53.138 has MTU less that 1 byte (physically unreal case) — it should be «routed» to a default class (yes, 1:fe49 is the default class for my shaper now). If this packet has a larger MTU — it should be dropped.

Therefore, our queueing discipline have not to cope with these packets (HTB in my case, but it may be any qdisc).

“Tweaking” photographed text pages

This time we have this task: we need to print badly taken digital pictures, so we have to improve them in some way — raise contrast, sharpen, make them «looking better».

As always, this result can be achieved going both easy (easy-to-use) and hard (where-am-i?..) way — we will pass both.

(Surely, below is not a single algorithm to do this job.)

Contents

  1. «Easy» — graphical — way
  2. «Hard» — from a command-line
  3. Conclusion

«Easy» way

  1. Launch your favotire graphical editor and open image:
    01-open-image
    If your camera give you horizontal image, you need to rotate it, of course.
  2. Duplicate background in a new layer:
    02-duplicate-layer
  3. Blur newly created (upper) layer a lot:
    03-blur-layer
    I’ve choosen value of 20 for blur radius, but this may be increased — have a try. I’ve been using up to 250 for some particular (rather large) images.
  4. Select «Division» as a composition mode for this layer:
    04-compose-divide
    After these steps we have:
    05-composed-result
  5. Flatten image:
    06-flatten
  6. Increase sharpness using «Unsharp mask» filter:
    07-unsharp-mask
  7. Save, print, enjoy…

In a real life case we may want to use some different algorithm, different values, but… we are talking about something completely different here :-)

Keep reading.

«Hard» way

image=${1:?"Set image name as a param"}
 
# let's say all images have three-letters "extension"
 
convert -monitor $image \
        \( +clone -blur 0x20 \) +swap \
        -compose divide -composite \
        -sharpen 0x1.3 ${image%%.*}_better.${image: -3:3}

Page before and page after (downsized for web).

Btw, if your camera gives horizontal image sometimes and you need to rotate some images — some, but in the same direction — you can use key «-rotate "90>"». This will rotate only horizontal images. Well, if you camera, luckily, provide your images with orientation tag, you can use -auto-orient, of course!

Btw, again, if you need to have black colour a bit more black, you can modify a script in, let’s say, this way:

image=${1:?"Set image name as a param"}
 
convert -monitor $image \
        -rotate "90>" \
        \( +clone -blur 0x20 \) +swap \
        -compose divide -composite \
        \( +clone -blur 20 \) +swap \
        -compose multiply -composite \
        -sharpen 0x1.3 ${image%%.*}_farbetter.${image: -3:3}

Page before, after the first script, after the second one (downsized for web). Again, this is not the only way, as if working with The GIMP or PhotoShop, of course :О)

For these particular images i’ve stopped on this:

image=${1:?"Set image name as a param"}
out="002"; mkdir -p ${out}
 
convert -monitor $image \
        -resize "3456x3456>" \
        -rotate "90>" \
        -median 1 \
        \( +clone -blur 0x20 \) +swap \
        -compose divide -composite \
        \( +clone -median 3 \) +swap \
        -compose multiply -composite \
        -level 10%,90%,1 \
        -sharpen 0x1.3 ${out}/${image%%.*}.${image: -3:3}

For the last picture this gives (downsized for web):
dscf3185_probably_final

Image is quite clean, occupies less disk space, can be pretty-printed :-)

Conclusion

:-)

I hope, you’ve agreed with my sarcasm.

And needless to say how easy that «hard» way — for a single picture, for all pictures in a directory, for all images around… Btw, long live „find” :-)

Of course, in real life it’s more convenient to have all processed images in a separate directory — it’s not a problem, too.

This command-line way works great under both Linux and (almost great,-) Windows. Just download and install ImageMagick. And read the manual :-)

Finally: my django-based configurator for HTB shaper works

Well.. I’ve done my django based configurator for my HTB shaper (bridge/linux).

I will try to translate this post in english; if you can read Ukrainian, try here.

Pages

Recent Posts

Most Rated

Highest Rated

Tags

Archives