Greg Kroah-Hartmann, a leader in the linux community, visited my CS411 class last Friday, May 23rd.

He works for Novell in the SUSE Labs division, based out of Nurumberg, Germany. He said only one person works there in Germany while everyone else works around the world.

He explained how the hierarchy of the people who develop linux works. It’s very efficient because at some point a file will have been touched by at least two people.

Linus Torvalds and Andrew Morton used to pull the subsystem trees together by themselves, but now they use something better called Linux NEXT.

I asked him what kind of source control program was his favorite to which he replied, “git”.

There are some stagnated areas in the kernel development like PCMCIA and serial ports, but then again some older ports have been really active.

One thing I liked to hear was that he said, “The most linux kernel developers in the world are in Portland.” so I’m not too far away from some good innovators.

Greg on wikipedia

No tags

Another CS411 project finished! This time around we implemented an I/O scheduler for Linux. What is an I/O scheduler you may ask? Well, it simply manages requests usually from a block device (e.g. hard disk, media drive) and not a character device. An I/O scheduler needs to be fair and give a share of the disk bandwidth to each running process.

With SSTF, requests are serviced based on their proximity to the disk head, with requests closest to the disk head being serviced first. Requests are kept in a queue. The SSTF algorithm looks at the current disk head position and iterates through the queue looking for the request that is the shortest physical distance away. That request is the one that gets serviced next, and the process continues.

Is this the best I/O scheduling algorithm? No. I’m afraid not. If there is a request that was physically very far from the disk head it would possibly never get serviced. This dilemma is known as “starvation”.

More efficient algorithms use an “elevator” approach wheras the disk head moves along the disk surface picking up requests (people on different floors) and when it gets to the end of the disk (top floor) it can turn around and pick up more people (more requests).

We used the built-in linked list structure provided by linux. Many helpful functions were found in list.h.

Here is a good article that discusses changes in the 2.6 kernel from linuxjournal.com.

No tags

A recent project I completed in my Operating Systems II class was to improve the existing linux SLOB memory allocator from first-fit to best-fit. The kernel currently gives 3 choices of allocators (SLAB, SLUB, and SLOB). The SLOB (Simple List Of Blocks) allocator, is designed to be a small and efficient allocation framework for use in small systems such as embedded systems. Unfortunately, a major limitation of the SLOB allocator is that it suffers greatly from internal fragmentation.

There is a recent reply from Linus Torvalds himself on a Linux mailing list where he suggested the use of a best-fit algorithm.

The first-fit uses the first available space for memory. The best-fit algorithm my team implemented works by looping through the list of free pages until the first page found with enough free space. Then, we sort the list from smallest to largest free space. In this way the first space encountered by the algorithm will be the smallest space which fits the required amount available. Essentially, we are modifying the input to the algorithm to ensure better fragmentation rates and utilizing space on each page better.

There wasn’t a lot lines of code to add. It was a fairly simple hack, but I’m sure it could be even more efficient.
We compared the fragmentation from both algorithms and there was a definite difference.

The version of Linux we used was 2.6.23.17

No tags

Vista showed a seemingly large upgrade in file copy improvement, at least technically. The end user may not have thought so though, “Perhaps the biggest drawback of the algorithm, and the one that has caused many Vista users to complain, is that for copies involving a large group of files between 256KB and tens of MB in size, the perceived performance of the copy can be significantly worse than on Windows XP”.

Mark Russinovich, a software engineer from Microsoft, explains what the internals of file copying in XP and shows what improvements were made for Vista and Vista Service Pack 1.

http://blogs.technet.com/markrussinovich/archive/2008/02/04/2826167.aspx

No tags

3:00

3:05

3:10

3:15

3:20

3:25

3:30

Look in fridge. Out of milk.

Leave for store.

Arrive at store.

Leave store.

Arrive home, put milk away.

Look in fridge.  Out of milk.

Leave for store.

Arrive at store.

Leave store.

Arrive home.  OH NO!

Mutual exclusion: Mechanisms that ensure that only one person or process is doing certain things at one time (others are excluded). E.g. only one person goes shopping at a time.

Critical section: A section of code, or collection of operations, in which only one process may be executing at a given time. E.g. shopping. It is a large operation that we want to make “sort of” atomic.

Solutions?

No tags

A thread in computer science is short for a thread of execution. Threads are a way for a program to fork (or split) itself into two or more simultaneously (or pseudo-simultaneously) running tasks. Threads and processes differ from one operating system to another but, in general, a thread is contained inside a process and different threads of the same process share some resources while different processes do not.

(more…)

No tags

No tags

In computer storage technology, a page is a fixed length block of memory that is used as a unit of transfer between physical memory and external storage like a disk, and a page fault is an interrupt (or exception) to the software raised by the hardware, when a program accesses a page that is mapped in address space, but not loaded in physical memory.

The hardware that detects this situation is the memory management unit in a processor. The exception handling software that handles the page fault is generally part of an operating system. The operating system tries to handle the page fault by making the required page accessible at a location in physical memory or kills the program in case it is an illegal access.

Page faults, by their very nature, degrade the performance of a program or operating system and in the degenerate case can cause thrashing. Optimizations to programs and the operating system that reduce the number of page faults that occur improve the performance of the program or even the entire system. The two primary focuses of the optimization effort focus on reducing overall memory usage and improving memory locality. Generally, making more physical memory available also reduces page faults. Many page replacement algorithms have been proposed, implementing a heuristic to reduce the incidence of page faults.

No tags

Demand paging is an application of virtual memory. In a system that uses demand paging, the operating system copies a disk page into physical memory only if an attempt is made to access it (i.e., if a page fault occurs). It follows that a process begins execution with none of its pages in physical memory, and many page faults will occur until most of a process’s working set of pages is located in physical memory. This is an example of lazy loading techniques.

No tags

A page, memory page, or virtual page is a fixed-length block of main memory, that is contiguous in both physical memory addressing and virtual memory addressing. A page is usually a smallest unit of data for:

• memory allocation performed by operating system for a program,
• transfer between main memory and any other auxiliary store, such as hard disk drive.

Virtual memory abstraction allows a page that does not currently reside in main memory to be addressed and used. If a program tries to access a location in such page, it generates an exception called page fault. The hardware or operating system is notified and loads the required page from auxiliary store automatically. A program addressing the memory has no knowledge of a page fault or a process following it. In consequence a program may be easily allowed to address more RAM than actually exists in the computer.

No tags