code cat

Kernel Debugging Tricks

Debugging the kernel is not necessarily rocket science; in fact it can be achieved using very simple and straight forward techniques and some time, patience and perseverance. This page describes some tricks and techniques to help debug the kernel.

printk is your friend

The simplest, and probably most effective way to debug the kernel is via printk(). This enables one to print messages to the console, and it very similar to printf(). Note that printk() can slow down the execution of code which can alter the way code runs, for example, changing the way race conditions occur.

CHANGING THE RING BUFFER SIZE

The internal kernel console message buffer can sometimes be too small to capture all of the printk messages, especially when debug code generates a lot of printk messages. If the buffer fills up, it wraps around and one can lose valueable debug messages.

To increase the internal buffer, use the kernel boot parameter:

log_buf_len=N

where N is the size of the buffer in bytes, and must be a power of 2.

CHANGING DEBUG LEVELS

One can specify the type of printk() log level by pre-pending the 1st printk() argument with one of the following:

KERN_EMERG    /* system is unusable                   */
KERN_ALERT    /* action must be taken immediately     */
KERN_CRIT     /* critical conditions                  */
KERN_ERR      /* error conditions                     */
KERN_WARNING  /* warning conditions                   */
KERN_NOTICE   /* normal but significant condition     */
KERN_INFO     /* informational                        */
KERN_DEBUG    /* debug-level messages                 */

e.g. printk(KERN_DEBUG "example debug message\n");

If one does not specify the log level then the default log level of KERN_WARNING is used. For example, enable all levels of console message:

echo 7 > /proc/sys/kernel/printk

To view console messages at boot, remove the quite and splash boot parameters from the kernel boot line in grub. This will disable the usplash splash screen and re-enable console messages.

Serial Console

Serial console enables one to dump out console messages over a serial cable. Most modern PCs do not have legacy serial ports, so instead, one can use a USB serial dongle instead. A "null serial cable" or "universal file transfer cable" is needed to connect the target computer with the host. Most commonly this will be a DB9 female to DB9 female null serial cable. In addition, one needs to enable USB serial support as a kernel build configuration:

CONFIG_USB_SERIAL_CONSOLE=y
CONFIG_USB_SERIAL=y

and enable the appropriate driver, e.g.:

CONFIG_USB_SERIAL_PL2303=y

and boot this kernel with

console=tty console=ttyUSB0,9600n8

one may need to adjust the baud rate appropriately.

Note: Generally, there is NO hardware or software flow control on serial console drivers, which means one may get dropped characters when running very high speed tty baud rates, such as 115200 baud.

Console Messages

Kernel Oops messages general contain a fair amount of information, ranging from register and process state dump and a stack dump too. Unfortunately the stack dump can be more than 25 lines and can scroll off the top of the 25 line Virtual Console. Hence to capture more of a Oops, try the following:

chvt 1
setfont /usr/share/consolefonts/Uni1-VGA8.psf.gz

Of course, one may still have a stack dump that scrolls the top of the Oops message off the console, so one trick is to rebuild the kernel with the stack dump removed, just to capture the initial Oops information. To do this, modify dump_stack in arch/x86/kernel/dumpstack_*.c and comment out the call to show_trace()

Slowing down kernel messages on boot

One may find a machine hangs during the kernel boot process and one would like to be able to see all the kernel messages but unfortunately they scroll off the console too quickly. One can slow down kernel console messages at boot time using by building the kernel with the following option enabled:

CONFIG_BOOT_PRINTK_DELAY=y

And boot the machine with the following kernel boot parameter:

boot_delay=N

where N = msecs delay between each console message.

Kernel panic during suspend

Debugging suspend/resume issues can be difficult if the kernel panics during suspend, especially late in the suspend because console messages are disabled. One can stop console messages from being suspended by using the kernel parameter no_console_suspend:

no_console_suspend=1

This will force the console not to suspend. Boot with this option, chvt 1 (to console #1), and suspend using pm-suspend

Serial Console in VirtualBox

In some debug scenerios it can be helpful to debug the kernel running inside a virtual machine. This is useful for some classes of non-hardware specific bugs, for example generic kernel core problems or debugging file system drivers.

One can capture Linux console messages running inside VirtualBox by setting it the VirtualBox serial log to /tmp/vbox and running a serial tty communications program such as minicom, and configure it to communicate with a named pipe tty called unix#/tmp/vbox

Boot with virtualised kernel boot line:

console=tty console=ttyS0,9600 

and minicom will capture the console messages

Network Console

One can route console messages over a network using netconsole. Note that it's not useful for capturing kernel panics as kernel halts before the messages can be transmitted over the network. However it can be useful to monitor systems without the need of message serial console cabling.

see Documentation/networking/netconsole.txt

netconsole=[src-port]@[src-ip]/[<dev>],[tgt-port]@<tgt-ip>/[tgt-macaddr]

        where
             src-port      source for UDP packets (defaults to 6665)
             src-ip        source IP to use (interface address)
             dev           network interface (eth0)
             tgt-port      port for logging agent (6666)
             tgt-ip        IP address for logging agent
             tgt-macaddr   ethernet MAC address for logging agent (broadcast)

Examples:

linux netconsole=4444@10.0.0.1/eth1,9353@10.0.0.2/12:34:56:78:9a:bc

The remote host can run either 'netcat -u -l -p <port>' or syslogd.

gdb on vmlinux

One can disassemble a built kernel using gdb on the vmlinux image. This is useful when one gets a kernel Oops message and a stack dump - one can then disassemble the object code and see where the Oops is occuring. For example:

gdb  debian/build/build-generic/vmlinux
(gdb) disassemble printk
Dump of assembler code for function printk:
0xffffffff8023dce0 <printk+0>:  sub    $0xd8,%rsp
0xffffffff8023dce7 <printk+7>:  lea    0xe0(%rsp),%rax
0xffffffff8023dcef <printk+15>: mov    %rsi,0x28(%rsp)
0xffffffff8023dcf4 <printk+20>: mov    %rsp,%rsi
0xffffffff8023dcf7 <printk+23>: mov    %rdx,0x30(%rsp)
0xffffffff8023dcfc <printk+28>: mov    %rcx,0x38(%rsp)
0xffffffff8023dd01 <printk+33>: mov    %rax,0x8(%rsp)
0xffffffff8023dd06 <printk+38>: lea    0x20(%rsp),%rax
0xffffffff8023dd0b <printk+43>: mov    %r8,0x40(%rsp)
0xffffffff8023dd10 <printk+48>: mov    %r9,0x48(%rsp)
0xffffffff8023dd15 <printk+53>: movl   $0x8,(%rsp)
0xffffffff8023dd1c <printk+60>: movl   $0x30,0x4(%rsp)
0xffffffff8023dd24 <printk+68>: mov    %rax,0x10(%rsp)
0xffffffff8023dd29 <printk+73>: callq  0xffffffff8023d980 <vprintk>
0xffffffff8023dd2e <printk+78>: add    $0xd8,%rsp
0xffffffff8023dd35 <printk+85>: retq   
End of assembler dump.

Objdump

If one has the built object code at hand, one can disassemble the object using objdump as follows:

objdump -SdCg debian/build/build-generic/fs/dcache.o

Using GDB to find the location where your kernel panicked or oopsed.

A quick and easy way to find the line of code where your kernel panicked or oopsed is to use GDB list command. You can do this as follows.

Lets assume your panic/oops message says something like:

[  174.507084] Stack:
[  174.507163]  ce0bd8ac 00000008 00000000 ce4a7e90 c039ce30 ce0bd8ac c0718b04 c07185a0
[  174.507380]  ce4a7ea0 c0398f22 ce0bd8ac c0718b04 ce4a7eb0 c037deee ce0bd8e0 ce0bd8ac
[  174.507597]  ce4a7ec0 c037dfe0 c07185a0 ce0bd8ac ce4a7ed4 c037d353 ce0bd8ac ce0bd8ac
[  174.507888] Call Trace:
[  174.508125]  [<c039ce30>] ? sd_remove+0x20/0x70
[  174.508235]  [<c0398f22>] ? scsi_bus_remove+0x32/0x40
[  174.508326]  [<c037deee>] ? __device_release_driver+0x3e/0x70
[  174.508421]  [<c037dfe0>] ? device_release_driver+0x20/0x40
[  174.508514]  [<c037d353>] ? bus_remove_device+0x73/0x90
[  174.508606]  [<c037bccf>] ? device_del+0xef/0x150
[  174.508693]  [<c0399207>] ? __scsi_remove_device+0x47/0x80
[  174.508786]  [<c0399262>] ? scsi_remove_device+0x22/0x40
[  174.508877]  [<c0399324>] ? __scsi_remove_target+0x94/0xd0
[  174.508969]  [<c03993c0>] ? __remove_child+0x0/0x20
[  174.509060]  [<c03993d7>] ? __remove_child+0x17/0x20
[  174.509148]  [<c037b868>] ? device_for_each_child+0x38/0x60
[  174.509241]  [<c039938f>] ? scsi_remove_target+0x2f/0x60
[  174.509393]  [<d0c38907>] ? __iscsi_unbind_session+0x77/0xa0 [scsi_transport_iscsi]
[  174.509699]  [<c015272e>] ? run_workqueue+0x6e/0x140
[  174.509801]  [<d0c38890>] ? __iscsi_unbind_session+0x0/0xa0 [scsi_transport_iscsi]
[  174.509977]  [<c0152888>] ? worker_thread+0x88/0xe0
[  174.510047]  [<c01566a0>] ? autoremove_wake_function+0x0/0x40

Lets say you want to know what line of code represents sd_remove+0x20/0x70. cd to the ubuntu debian/build/build-generic directory in your kernel tree and run gdb on the ".o" file which has the function sd_remove() in this case in sd.o, and use the gdb "list" command, (gdb) list *(function+0xoffset), in this case function is sd_remove() and offset is 0x20, and gdb should tell you the line number where you hit the panic or oops. This has worked for me very reliably for most cases.

manjo@hungry:~/devel/ubuntu/kernel/ubuntu-karmic-397906/debian/build/build-generic/drivers/scsi$ gdb sd.o
GNU gdb (GDB) 6.8.50.20090628-cvs-debian
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
(gdb) list *(sd_remove+0x20)
0x1650 is in sd_remove (/home/manjo/devel/ubuntu/kernel/ubuntu-karmic-397906/drivers/scsi/sd.c:2125).
2120    static int sd_remove(struct device *dev)
2121    {
2122            struct scsi_disk *sdkp;
2123    
2124            async_synchronize_full();
2125            sdkp = dev_get_drvdata(dev);
2126            blk_queue_prep_rq(sdkp->device->request_queue, scsi_prep_fn);
2127            device_del(&sdkp->dev);
2128            del_gendisk(sdkp->disk);
2129            sd_shutdown(dev);
(gdb)

BSD 와 리눅스 배포판에서는 /etc/rsyslog.conf 파일을 사용하는데, rsyslog.conf는 syslog(결국 rsyslogd)의 설정 파일이다.

그리고 local7은 0~7 로 나눠진 user defined local facility이며, 특정 daemon에 대한 로깅을 하기 위한 수단이라고 생각하면 된다.

dentry cache hash table size 지정은,
fs/dcache.c에서 alloc_large_system_hash를 통해서 된다.  그전에 dhash_entries는 kernel cmdline에서 지정하여 원하는 값을 패스 할 수 있다.
static void __init dcache_init(void)
{
        int loop;

        /* 
         * A constructor could be added for stable state like the lists,
         * but it is probably not worth it because of the cache nature
         * of the dcache. 
         */
        dentry_cache = KMEM_CACHE(dentry,
                SLAB_RECLAIM_ACCOUNT|SLAB_PANIC|SLAB_MEM_SPREAD);

        /* Hash may have been set up in dcache_init_early */
        if (!hashdist)
                return;

        dentry_hashtable =
                alloc_large_system_hash("Dentry cache",
                                        sizeof(struct hlist_bl_head),
                                        dhash_entries,
                                        13,
                                        0,
                                        &d_hash_shift,
                                        &d_hash_mask,
                                        0);

        for (loop = 0; loop < (1 << d_hash_shift); loop++)
                INIT_HLIST_BL_HEAD(dentry_hashtable + loop);
}

그리고 만약 dhash_entries값이 0이면(전역비초기화 변수 이므로 0으로 되어 bss영역에 있을 것이다) 5395라인 하위로 타고 들어가, nr_kernel_pages에 기반을 둔 2의 배수를 설정하고 기타 log2작업(아 이건 나중에 보자...)을 통해 hash table size를 출력한다. 

void *__init alloc_large_system_hash(const char *tablename,
                                     unsigned long bucketsize,
                                     unsigned long numentries,
                                     int scale,
                                     int flags,
                                     unsigned int *_hash_shift,
                                     unsigned int *_hash_mask,
                                     unsigned long limit)
{
        unsigned long long max = limit;
        unsigned long log2qty, size;
        void *table = NULL;

        /* allow the kernel cmdline to have a say */
        if (!numentries) {
                /* round applicable memory size up to nearest megabyte */
                numentries = nr_kernel_pages;
                numentries += (1UL << (20 - PAGE_SHIFT)) - 1;
                numentries >>= 20 - PAGE_SHIFT;
                numentries <<= 20 - PAGE_SHIFT;

                /* limit to 1 bucket per 2^scale bytes of low memory */
                if (scale > PAGE_SHIFT)
                        numentries >>= (scale - PAGE_SHIFT);
                else
                        numentries <<= (PAGE_SHIFT - scale);

                /* Make sure we've got at least a 0-order allocation.. */
                if (unlikely(flags & HASH_SMALL)) {
                        /* Makes no sense without HASH_EARLY */
                        WARN_ON(!(flags & HASH_EARLY));
                        if (!(numentries >> *_hash_shift)) {
                                numentries = 1UL << *_hash_shift;
                                BUG_ON(!numentries);
                        }
                } else if (unlikely((numentries * bucketsize) < PAGE_SIZE))
                        numentries = PAGE_SIZE / bucketsize;
        }
        numentries = roundup_pow_of_two(numentries);

        /* limit allocation size to 1/16 total memory by default */
        if (max == 0) {
                max = ((unsigned long long)nr_all_pages << PAGE_SHIFT) >> 4;
                do_div(max, bucketsize);
        }

        if (numentries > max)
                numentries = max;

        log2qty = ilog2(numentries);

        do {
                size = bucketsize << log2qty;
                if (flags & HASH_EARLY)
                        table = alloc_bootmem_nopanic(size);
                else if (hashdist)
                        table = __vmalloc(size, GFP_ATOMIC, PAGE_KERNEL);
                else {
                        /*
                         * If bucketsize is not a power-of-two, we may free
                         * some pages at the end of hash table which
                         * alloc_pages_exact() automatically does
                         */
                        if (get_order(size) < MAX_ORDER) {
                                table = alloc_pages_exact(size, GFP_ATOMIC);
                                kmemleak_alloc(table, size, 1, GFP_ATOMIC);
                        }
                }
        } while (!table && size > PAGE_SIZE && --log2qty);

        if (!table)
                panic("Failed to allocate %s hash table\n", tablename);

        printk(KERN_INFO "%s hash table entries: %ld (order: %d, %lu bytes)\n",
               tablename,
               (1UL << log2qty),
               ilog2(size) - PAGE_SHIFT,
               size);

        if (_hash_shift)
                *_hash_shift = log2qty;
        if (_hash_mask)
                *_hash_mask = (1 << log2qty) - 1;

        return table;
}

arch/arm/mm/init.c : mem_init(void)

        unsigned long reserved_pages, free_pages;
        struct memblock_region *reg;

 /*
         * Since our memory may not be contiguous, calculate the
         * real number of pages we have in this system
         */
        printk(KERN_INFO "Memory:");
        num_physpages = 0;
        for_each_memblock(memory, reg) {
                unsigned long pages = memblock_region_memory_end_pfn(reg) -
                        memblock_region_memory_base_pfn(reg);
                num_physpages += pages;
                printk(" %ldMB", pages >> (20 - PAGE_SHIFT));
        }
        printk(" = %luMB total\n", num_physpages >> (20 - PAGE_SHIFT));

        printk(KERN_NOTICE "Memory: %luk/%luk available, %luk reserved, %luK highmem\n",
                nr_free_pages() << (PAGE_SHIFT-10),
                free_pages << (PAGE_SHIFT-10),
                reserved_pages << (PAGE_SHIFT-10),
                totalhigh_pages << (PAGE_SHIFT-10));

include/linux/memblock.h

#define for_each_memblock(memblock_type, region)                                        \
        for (region = memblock.memblock_type.regions;                           \
             region < (memblock.memblock_type.regions + memblock.memblock_type.cnt);    \
             region++)

include/linux/memblock.h

/**
 * memblock_region_memory_end_pfn - Return the end_pfn this region
 * @reg: memblock_region structure
 */
static inline unsigned long memblock_region_memory_end_pfn(const struct memblock_region *reg)
{
        return PFN_DOWN(reg->base + reg->size);
}

include/linux/memblock.h

/**
 * memblock_region_reserved_base_pfn - Return the lowest pfn intersecting with the reserved region
 * @reg: memblock_region structure
 */
static inline unsigned long memblock_region_reserved_base_pfn(const struct memblock_region *reg)
{
        return PFN_DOWN(reg->base);
}

include/linux/memblock.h

struct memblock {
        phys_addr_t current_limit;
        phys_addr_t memory_size;        /* Updated by memblock_analyze() */
        struct memblock_type memory;
        struct memblock_type reserved;
};

meminfo 라는 구조체에 nr_banks 와 membank라는 구조체가 멤버로 있다.  이들은 부팅 과정에서 이미 arm_add_memory라는 함수를 통해서 bank에 대한 start 및 size가 정해졌있다.(
early_init_dt_add_memory()나 parse_tag_mem32()등의 함수를 통해서 세팅된다(확인필요)

struct membank {
        phys_addr_t start;
        phys_addr_t size;
        unsigned int highmem;
};

struct meminfo {
        int nr_banks;
        struct membank bank[NR_BANKS];
};

그리고 arch/arm/mm/init.c:find_limits()라는 함수를 통해서, 각 메모리 뱅크를 돌면서 (참고로 이건 3.1.10소스다. 최신은 좀 틀리다)

static void __init find_limits(unsigned long *min, unsigned long *max_low,
        unsigned long *max_high)
{
        struct meminfo *mi = &meminfo;
        int i;

        *min = -1UL;
        *max_low = *max_high = 0;

        for_each_bank (i, mi) {
                struct membank *bank = &mi->bank[i];
                unsigned long start, end;

                start = bank_pfn_start(bank);
                end = bank_pfn_end(bank);

                if (*min > start)
                        *min = start;
                if (*max_high < end)
                        *max_high = end;
                if (bank->highmem)
                        continue;
                if (*max_low < end)
                        *max_low = end;
        }
}

밑의 매크로를 통해서 페이지프레임넘버를 가져온다.  

#define bank_pfn_start(bank)    __phys_to_pfn((bank)->start)
#define bank_pfn_end(bank)      __phys_to_pfn((bank)->start + (bank)->size)

#define __phys_to_pfn(paddr)    ((unsigned long)((paddr) >> PAGE_SHIFT))

가져온 pfn을 start, end에 assign하고 전달받은 min, max_low, max_high를 그에 맞게 설정해 준다.  

그 후, bootmem_init()에서 arm_bootmem_init(min, max_low)를 통해 bootmem bitmap page 할당, bootmem allocator 초기화, lowmem 해재, lowmem memblock reservation 등을 한다.

위의 할당된 bootmem 은 나중에 init/main.c 에서 mem_init()를 호출하면,

static void __init mm_init(void)
{
        /*
         * page_cgroup requires countinous pages as memmap
         * and it's bigger than MAX_ORDER unless SPARSEMEM.
         */
        page_cgroup_init_flatmem();
        mem_init();
        kmem_cache_init();
        percpu_init_late();
        pgtable_cache_init();
        vmalloc_init();
}

mem_init()에서 bootmem에 할당되었던 메모리를 해제한다.
 600 totalram_pages += free_all_bootmem();

mm/bootmem.c

unsigned long __init free_all_bootmem(void)
{
        unsigned long total_pages = 0;
        bootmem_data_t *bdata;

        list_for_each_entry(bdata, &bdata_list, list)
                total_pages += free_all_bootmem_core(bdata);

        return total_pages;
}

자 그러면 이제 우리가 원하던 부분이다.

        reserved_pages = free_pages = 0;

        for_each_bank(i, &meminfo) {
                struct membank *bank = &meminfo.bank[i];
                unsigned int pfn1, pfn2;
                struct page *page, *end;

                pfn1 = bank_pfn_start(bank);
                pfn2 = bank_pfn_end(bank);

                page = pfn_to_page(pfn1);
                end  = pfn_to_page(pfn2 - 1) + 1;

                do {
                        if (PageReserved(page))
                                reserved_pages++;
                        else if (!page_count(page))
                                free_pages++;
                        page++;
                } while (page < end);
        }

        /*
         * Since our memory may not be contiguous, calculate the
         * real number of pages we have in this system
         */
        printk(KERN_INFO "Memory:");
        num_physpages = 0;
        for_each_memblock(memory, reg) {
                unsigned long pages = memblock_region_memory_end_pfn(reg) -
                        memblock_region_memory_base_pfn(reg);
                num_physpages += pages;
                printk(" %ldMB", pages >> (20 - PAGE_SHIFT));
        }
        printk(" = %luMB total\n", num_physpages >> (20 - PAGE_SHIFT));

        printk(KERN_NOTICE "Memory: %luk/%luk available, %luk reserved, %luK highmem\n",
                nr_free_pages() << (PAGE_SHIFT-10),
                free_pages << (PAGE_SHIFT-10),
                reserved_pages << (PAGE_SHIFT-10),
                totalhigh_pages << (PAGE_SHIFT-10));

#define MLK(b, t) b, t, ((t) - (b)) >> 10
#define MLM(b, t) b, t, ((t) - (b)) >> 20
#define MLK_ROUNDUP(b, t) b, t, DIV_ROUND_UP(((t) - (b)), SZ_1K)

        printk(KERN_NOTICE "Virtual kernel memory layout:\n"
                        "    vector  : 0x%08lx - 0x%08lx   (%4ld kB)\n"
#ifdef CONFIG_HAVE_TCM
                        "    DTCM    : 0x%08lx - 0x%08lx   (%4ld kB)\n"
                        "    ITCM    : 0x%08lx - 0x%08lx   (%4ld kB)\n"
#endif
                        "    fixmap  : 0x%08lx - 0x%08lx   (%4ld kB)\n"
#ifdef CONFIG_MMU
                        "    DMA     : 0x%08lx - 0x%08lx   (%4ld MB)\n"
#endif
                        "    vmalloc : 0x%08lx - 0x%08lx   (%4ld MB)\n"
                        "    lowmem  : 0x%08lx - 0x%08lx   (%4ld MB)\n"
#ifdef CONFIG_HIGHMEM
                        "    pkmap   : 0x%08lx - 0x%08lx   (%4ld MB)\n"
#endif
                        "    modules : 0x%08lx - 0x%08lx   (%4ld MB)\n"
                        "      .text : 0x%p" " - 0x%p" "   (%4d kB)\n"
                        "      .init : 0x%p" " - 0x%p" "   (%4d kB)\n"
                        "      .data : 0x%p" " - 0x%p" "   (%4d kB)\n"
                        "       .bss : 0x%p" " - 0x%p" "   (%4d kB)\n",

meminfo의 nr_bank숫자 만큼 루핑을 하면서, bank의 start주소와 end주소의 pfn을 가져와, 각각 page, end에 저장시키고, page에서부터 end까지 안에서 루핑하면서, page가 reserved일 경우, reserved_pages++ 하고, !page_count(page)일 경우 free_pages++하고, 마지막으로 page를 increment시킨다.

 for_each_memblock(memory, reg) 는 

#define for_each_memblock(memblock_type, region)                                        \
        for (region = memblock.memblock_type.regions;                           \
             region < (memblock.memblock_type.regions + memblock.memblock_type.cnt);    \
             region++)

memblock_type의 memory안의 memblock_region의 regions을 돌면서(memory의 cnt숫자만큼), 

struct memblock_region {
        phys_addr_t base;
        phys_addr_t size;
};

struct memblock_type {
        unsigned long cnt;      /* number of regions */
        unsigned long max;      /* size of the allocated array */
        struct memblock_region *regions;
};

struct memblock {
        phys_addr_t current_limit;
        phys_addr_t memory_size;        /* Updated by memblock_analyze() */
        struct memblock_type memory;
        struct memblock_type reserved;
};

pages에 region의 base + size의 PFN 값에서 base의 PFN값을 뺀 값을 설정한다.

                unsigned long pages = memblock_region_memory_end_pfn(reg) -
                        memblock_region_memory_base_pfn(reg);

그리고 이렇게 얻어진 pages를 num_phypages에 증가시켜준다.

그러면 드디오 이 num_phypages >> (20 - PAGE_SHIFT)가
Memory: = xxx MB total

에 나오는 xxx가 된다. 

driver초기화 시에 흔히 불리는 module_init(x)는 include/linux/init.h에

로 정의되어 있다.  여기서 x는 kernel 부팅 타임이나 모듈 삽입시에 불리는 함수명이 된다.

built in 모듈일 경우, do_initcalls()에서 불릴 것이고,

module 일 경우, module insertion에 불린다.

MACH_TYPE_XXX 와 machine 이름을 overwrite하기 위해서는 아래 매크로를 활용하면 된다.

linux/arch/arm/include/asm/mach/arch.h

 * Set of macros to define architecture features.  This is built into
 * a table by the linker.
 */
#define MACHINE_START(_type,_name)                      \
static const struct machine_desc __mach_desc_##_type    \
 __used                                                 \
 __attribute__((__section__(".arch.info.init"))) = {    \
        .nr             = MACH_TYPE_##_type,            \
        .name           = _name,

#define MACHINE_END                             \
};

start_kernel()-->setup_arch()-->setup_machine_tags()

arch/arm/kernel/setup.c에서setup_arch()함수 내에서 머신이름을 설정한다.

setup_arch(char **cmdline_p) { struct machine_desc *mdesc; mdesc = setup_machine_fdt(__atgs_pointer); if (!mdesc) mdesc = setup_machine_tags(machine_arch_type);

machine_arch_type은 setup_machine_tags의 인자로 넘어가고, setup_machine_tags()내의
for_each_machine_desc는 __arch_info_begin 에서 __arch_info_end 를 순환하면서 machine_desc구조체의 nr(architecture number를 인자로 받아온 machine_arch_type과 비교하여 같은 경우, archiecture name을 찍어준다.

static struct machine_desc * __init setup_machine_tags(unsigned int nr) { struct machine_desc *mdesc = NULL, *p; ... for_each_machine_desc(p) if (nr == p->nr) { printk("Machine: %s\n", p->name); mdesc = p; break; } }

early_param("HI_MEM", HI_MEM_Init) 

  --> __setup_param(str, fn, fn, 1) 으로 정의되어 있는데 이를 자세히 보면,

static const char __setup_str_##unique_id[] __initcost \
__aligned(1) = str; \
static struct obs_kernel_param __setup_##unique_id \
  __used __section(.init.setup)
  __attribute__((aligned((sizeof(long))))) \
  = { __setup_str_##unique_id, fn, early }

long 사이즈로 align된 .init.setp섹션에 obs_kernel_param 구조체 형태의 __setup_##unique_id 안에 { __setup_str_##unique_id, fn, early } 형태로 초기화 시킨다. 

obs_kernel_param 구조체는 

const char *str
int (*setup_func)(char *);
int early;

로 구성되어 있다.

이렇게 설정된 early_param의 내용은 init/main.c에서 parse_early_options() --> parse_args("early options", cmdline, NULL, 0, do_early_param) --> parse_one()안에서
return handle_unknown에서 do_early_param이 실행된다.

do_early_param을 실행시킬 때, 그동안 쌓인 obs_kernel_param 리스트(?)들을 순회하면서 실행 시킨다.(섹션에 구조체가 리스트로 들어가기 보다는 각각의 구조체가 엔트리로 들어가는 거 같은데 좀 더 확인이 필요하다.)

__setup_start[] 와 __setup_end[]는 include/asm-generic/vmlinux.lds.h 에서 다음과 같이 정의되어 있다.

#define INIT_SETUP(initsetup_align)                                     \
                . = ALIGN(initsetup_align);                             \
                VMLINUX_SYMBOL(__setup_start) = .;                      \
                *(.init.setup)                                          \
                VMLINUX_SYMBOL(__setup_end) = .;

또한 INIT_SETUP은 arch/arm/kernel/vmlinux.lds.S에서 16byte(?)으로 align되어 있다.

        .init.data : {
#ifndef CONFIG_XIP_KERNEL
                INIT_DATA
#endif
                INIT_SETUP(16)
                INIT_CALLS
                CON_INITCALL
                SECURITY_INITCALL
                INIT_RAM_FS
        }

좀 더 자세히 봐야 하는데, 결국 맨 위, "HI_MEM"이라는 parameter에 대한 처리를 HI_MEM_init에서 처리할 수 있게 된다.