linux-context

Linux中的上下文

基于ARM64

ARM64 exception level

ARM64 Exception level

  • Linux 内核运行于Normal world EL1
  • 目前手机上在EL1的Normal world只运行了一个Guest OS,Linux
  • Secure world运行的镜像是tz app, trustzone系统
  • EL2 目前用作虚拟化
  • EL3 执行资源保护的检查
    [//]: 上图仅适用于ARM64 i.e.AARCH64

Linux中的上下文

1. 硬中断 (上半部)

  1. 硬件中断中执行,发生中断后直接跳转到异常向量表执行
  2. 不同的中断之间可以相互打断
  3. 同一个中断,同一时刻只会在同一个CPU上运行

2. 软中断 (下半部)

  1. 在中断执行完成返回时,判断是否有软中断事物需要执行,如果有,就跳转执行
  2. 可以中断除了中断之外的其他上下文
  3. 共有以下几种级别
enum
{
    HI_SOFTIRQ=0,
    TIMER_SOFTIRQ,
    NET_TX_SOFTIRQ,
    NET_RX_SOFTIRQ,
    BLOCK_SOFTIRQ,
    IRQ_POLL_SOFTIRQ,
    TASKLET_SOFTIRQ,
    SCHED_SOFTIRQ,
    HRTIMER_SOFTIRQ, /* Unused, but kept as tools rely on the
                numbering. Sigh! */
    RCU_SOFTIRQ,    /* Preferable RCU should always be the last softirq */

    NR_SOFTIRQS
};
  1. 软中断个数,以及实现函数在编译时确定,运行时不能动态注册
  2. irq_handle->gic_handle_irq()->handle_domain_irq()->irq_exit() -> invoke_softirq()
    执行顺序:按照顺序一个一个执行
  3. 同一个softirq可以在多个cpu core上同时执行,所以需要加锁同步

3. tasklet

  1. 利用软中断实现的可以动态注册的底半部机制
  2. 两个softirq等级

+ tasklet_vec -------- TASKLET_SOFTIRQ --- tasklet_action
+ tasklet_hi_vec ------- HI_SOFTIRQ ------ tasklet_hi_action
3. 同一个tasklet只会在同一个cpu core中执行,不会同时调度到多个cpu core中执行

4. 原子上下文

  1. 在进程上下文中调用了spinlock, 禁止抢占,以及关闭中断的函数之后就属于原子上下文
  2. 原子上下文中不可以调用能使进程休眠的函数
  3. 上述的硬中断,softirq,tasklet都属于原子上下文

5. 进程上下文

  1. 普通的内核线程,普通进程

Linux 同步技术

Unreliable Guide To Locking

各种上下文中加锁方法

IRQ Handler A IRQ Handler B Softirq A Softirq B Tasklet A Tasklet B Timer A Timer B User Context A User Context B
IRQ Handler A None
IRQ Handler B SLIS None
Softirq A SLI SLI SL
Softirq B SLI SLI SL SL
Tasklet A SLI SLI SL SL None
Tasklet B SLI SLI SL SL SL None
Timer A SLI SLI SL SL SL SL None
Timer B SLI SLI SL SL SL SL SL None
User Context A SLI SLI SLBH SLBH SLBH SLBH SLBH SLBH None
User Context B SLI SLI SLBH SLBH SLBH SLBH SLBH SLBH MLI None
Table: Table of Locking Requirements
NAME lock_type
SLIS spin_lock_irqsave
SLI spin_lock_irq
SL spin_lock
SLBH spin_lock_bh
MLI mutex_lock_interruptible

Linux中各种锁

原子变量

  1. 原子操作是各种锁的基石
  2. 原子操作在同步中的作用
多个core之间竞争
多个进程之间竞争(why ?)
Instance 1 Instance 2
read very_important_count (5)
read very_important_count (5)
add 1 (6)
add 1 (6)
write very_important_count (6)
write very_important_count (6)

ARM64实现:
ARMv7-A and ARMv8-A architectures both provide support for exclusive memory accesses.
In A64, this is the Load/Store exclusive (LDXR/STXR) pair.

crash64> dis _raw_spin_lock
0xffffff8f41b5e4f8 <__cpuidle_text_end>:        mrs     x2, sp_el0
0xffffff8f41b5e4fc <_raw_spin_lock+4>:  ldr     w1, [x2,#16]
0xffffff8f41b5e500 <_raw_spin_lock+8>:  add     w1, w1, #0x1
0xffffff8f41b5e504 <_raw_spin_lock+12>: str     w1, [x2,#16]
0xffffff8f41b5e508 <_raw_spin_lock+16>: prfm    pstl1strm, [x0]
0xffffff8f41b5e50c <_raw_spin_lock+20>: ldaxr   w1, [x0] //独占的load
0xffffff8f41b5e510 <_raw_spin_lock+24>: add     w2, w1, #0x10, lsl #12
0xffffff8f41b5e514 <_raw_spin_lock+28>: stxr    w3, w2, [x0] //独占的store
0xffffff8f41b5e518 <_raw_spin_lock+32>: cbnz    w3, 0xffffff8f41b5e50c  //如果w3不是0,说明有其他的设备访问过[x0],这次的读改写操作需要重新开始
0xffffff8f41b5e51c <_raw_spin_lock+36>: eor     w2, w1, w1, ror #16
0xffffff8f41b5e520 <_raw_spin_lock+40>: cbz     w2, 0xffffff8f41b5e538
0xffffff8f41b5e524 <_raw_spin_lock+44>: sevl
0xffffff8f41b5e528 <_raw_spin_lock+48>: wfe
0xffffff8f41b5e52c <_raw_spin_lock+52>: ldaxrh  w3, [x0]
0xffffff8f41b5e530 <_raw_spin_lock+56>: eor     w2, w3, w1, lsr #16
0xffffff8f41b5e534 <_raw_spin_lock+60>: cbnz    w2, 0xffffff8f41b5e528
0xffffff8f41b5e538 <_raw_spin_lock+64>: ret

Memory barrier

  1. Data Memory Barrier (DMB). This forces all earlier-in-program-order memory accesses to become globally visible before any subsequent accesses. 会强制化使所有对内存的操作可以被下边的指令可见
  2. Data Synchronization Barrier (DSB). All pending loads and stores, cache maintenance instructions, and all TLB maintenance instructions, are completed before program execution continues. A DSB behaves like a DMB, but with additional properties. 加入了更多的tlb,cache相关flush操作,比DMB更强力
  3. Instruction Synchronization Barrier (ISB). This instruction flushes the CPU pipeline and prefetch buffers, causing instructions after the ISB to be fetched (or re-fetched) from cache or memory. flush流水线,重新装载流水线指令缓存。

例子:

LDR X0, [X3]
LDNP X2, X1, [X0] // Xo may not be loaded when the instruction executes!
To correct the above, you need an explicit load barrier:
LDR X0, [X3]
DMB nshld
LDNP X2, X1, [X0]

spinlock

crash64> whatis arch_spinlock_t
typedef struct {
    u16 owner;
    u16 next;
} arch_spinlock_t;

spinlock通过两个域实现,防止多cpu竞争而导致活锁
通过汇编实现以提高性能
Linux内核同步机制之(四):spin lock

C代码(仅参考实现,有些应该原子操作的各位看官自己心里有数即可):

_raw_spin_lock(arch_spinlock_t *lock){
    arch_spinlock_t local_lock;
    local_lock = *lock;
    lock.next++;
retry:
    if (lock.owner == local_lock.next)
        return ;
    wfe;
    goto retry;
}

rwlock

typedef struct {
    volatile unsigned int lock;
} arch_rwlock_t;

同步原语

  • 同时可以有多个执行体一起执行读操作
  • 一次只能有一个执行体执行写操作
  • 写操作必须等待读操作完成

lock值定义

| 31 | 30 0 |
|-|-|
| Write Thread Counter |Read Thread Counter|
Linux中常见同步机制设计原理

seqlock

typedef struct {
    struct seqcount seqcount;
    spinlock_t lock;
} seqlock_t;

typedef struct seqcount {
    unsigned sequence;
#ifdef CONFIG_DEBUG_LOCK_ALLOC
    struct lockdep_map dep_map;
#endif
} seqcount_t;

同步原语

  • 同时可以有多个执行体一起执行读操作
  • 一次只能有一个执行体执行写操作
  • 写操作无需等待读操作完成
  • 如果读操作被打断,需要重新开始读

seqcount值初始化为0,偶数代表有读者正在持有锁

void read(void)
{
       bool x, y;

       do {
               int s = read_seqcount_begin(&seq);

               x = X; y = Y;

         } while (read_seqcount_retry(&seq, s));
}

Tree RCU (非SRCU,原子上下文,使用RCU之后不可睡眠,不可被抢占)

同步原语

  • 无锁化操作,读写都不需要持有任何锁
  • 性能最高,但是适用范围小,适用复杂
int register_cxl_calls(struct cxl_calls *calls)
{
    if (cxl_calls)
        return -EBUSY;

    rcu_assign_pointer(cxl_calls, calls);
    return 0;
}
EXPORT_SYMBOL_GPL(register_cxl_calls);

void unregister_cxl_calls(struct cxl_calls *calls)
{
    BUG_ON(cxl_calls->owner != calls->owner);
    RCU_INIT_POINTER(cxl_calls, NULL);
    synchronize_rcu();
}
EXPORT_SYMBOL_GPL(unregister_cxl_calls);

两个概念

Grace Periodguan (GP)宽限期
  • 临界区开始到所有的CPU core都进入过一次QS的这段时期,在此时期内,认为RCU保护的旧对象不能释放
Quiescent State (QS)静止状态
  • 进行调度即说明进入了QS, rcu有bh, sched等,具体需要调用的函数,视保护变量适用的上下文而定

GP示意图

Mutex Lock

  1. Mutex Lock在获取锁失败时会让出CPU执行权,转而去执行其他程序
  2. Mutex Lock适用于需要长时间获取锁的情况
  3. Mutex Lock获取之后也可以发生进程切换以及睡眠
  4. 持锁,释放锁开销比较大

关于“linux-context”我的2个想法

  1. Hello there,

    My name is Aly and I would like to know if you would have any interest to have your website here at schspa.ml promoted as a resource on our blog alychidesign.com ?

    We are in the midst of updating our broken link resources to include current and up to date resources for our readers. Our resource links are manually approved allowing us to mark a link as a do-follow link as well
    .
    If you may be interested please in being included as a resource on our blog, please let me know.

    Thanks,
    Aly

发表评论

电子邮件地址不会被公开。 必填项已用*标注