Solaris Scheduling

About This Presentation

Title:

Solaris Scheduling

Description:

The Solaris kernel may be seen as a bundle of kernel threads ... threshhold/ 54 short ts_maxwait; / bumped to ts_lwait if more than ts_maxwait ... – PowerPoint PPT presentation

Number of Views:295

Avg rating:3.0/5.0

Slides: 47

Provided by: jinw2

Learn more at: https://lin-web.clarkson.edu

Category:

more less

Transcript and Presenter's Notes

Title: Solaris Scheduling

1
Solaris Scheduling

Bongio Jeremy
Wenjin Hu

2
Overview

Table Driven
Loadable class module
Thread-level scheduling
The Solaris kernel may be seen as a bundle of
kernel threads
A kernel thread is the entity that is scheduled
by the kernel
If no lightweight process is attached, it is also
known as a system thread
Kernel preemptable

3
Class and priority

Interrupt
global prio -gt 100109/160169
user prio -gt 0-9
Not a really scheduling class
Real Time
global prio -gt 100159
user prio -gt 0-59
SYS
global prio -gt 6099
user prio -gt 0-39
TimeShare
global prio -gt 059
user prio -gt 0-59
Interactive
sharesTS dispatch table

4
Priority Classes

Global Priority Scheme and Scheduling Classes (1)

5
Class and priority (Cont)

Two level priority
Systemwide-relative priority(Global Priority)
NOT tunable(Global Range)
Class-relative priority(Class Priority)
Tuned by the kernel/dispatcher (Adjustment
Rang)

6
/uts/common/sys/class.h

105 typedef struct sclass
106 char cl_name / class name /
107 / class specific initialization function /
108 pri_t (cl_init)(id_t, int, classfuncs_t
)
/scheduling-class-dependent(class_ops
thread_ops)/
/thread can enter the class/
109 classfuncs_t cl_funcs / pointer to
classfuncs structure /
/kernel lock for synchronized access to the
class structure/
110 krwlock_t cl_lock / class structure
read/write lock /
111 int cl_count / of threads trying to load
class /
112 sclass_t

7
SYS

Critical resource
Preemptable
NOT time sliced
Priority defined in a simple array
NOT loadable
In the TS framework
TS/IA may be temporarily adjust to SYS

8
/uts/common/disp/ts_dptbl.c

77 define TSGPUP0 0 / Global priority for TS
user priority 0 /
78 define TSGPKP0 60 / Global priority for TS
kernel priority 0 /
79
80 /
81 array of global priorities used by ts procs
sleeping or
82 running in kernel mode after sleep
83 /
84
85 pri_t config_ts_kmdpris
86 TSGPKP0, TSGPKP01, TSGPKP02, TSGPKP03,
87 TSGPKP04, TSGPKP05, TSGPKP06, TSGPKP07,
88 TSGPKP08, TSGPKP09, TSGPKP010,
TSGPKP011,
89 TSGPKP012, TSGPKP013, TSGPKP014,
TSGPKP015,
90 TSGPKP016, TSGPKP017, TSGPKP018,
TSGPKP019,
91 TSGPKP020, TSGPKP021, TSGPKP022,
TSGPKP023,
92 TSGPKP024, TSGPKP025, TSGPKP026,
TSGPKP027,
93 TSGPKP028, TSGPKP029, TSGPKP030,
TSGPKP031,
94 TSGPKP032, TSGPKP033, TSGPKP034,
TSGPKP035,
95 TSGPKP036, TSGPKP037, TSGPKP038,
TSGPKP039

9
Realtime

Capable of preempting SYS
Memory locking
Run at fixed priority but can be configured
(kernel cannot change its priority)

10
/src/uts/common/sys/rt.h

46 typedef struct rtdpent
47 pri_t rt_globpri / global (class
independent) priority /
48 int rt_quantum / default quantum associated
with this level /
49 rtdpent_t

11
/uts/common/disp/rt_dptbl.c

73 define RTGPPRIO0 100 / Global priority for
RT priority 0 /
74
75 rtdpent_t config_rt_dptbl
76
77 / prilevel Time quantum /
78
79 RTGPPRIO0, 100,
80 RTGPPRIO01, 100,
81 RTGPPRIO02, 100,
97 RTGPPRIO018, 80,
102 RTGPPRIO023, 60,
107 RTGPPRIO028, 60,
112 RTGPPRIO033, 40,
117 RTGPPRIO038, 40,
122 RTGPPRIO043, 20,
127 RTGPPRIO048, 20,
137 RTGPPRIO058, 10,
138 RTGPPRIO059, 10
139

12
Dispatcher Table

Contains default values for priority and
priortity readjustment
Get its global priority by user priority as index
Get quantum by its global priority
Indicate how to adjust the priority
TS example

13
/src/uts/common/sys/ts.h

47 typedef struct tsdpent
48 pri_t ts_globpri / global (class
independent) priority /
49 int ts_quantum / time quantum given to
procs at this level /
/favors IA or CPU-bound/
/parameters to calculate the class-relative
priority/
/deduct 10 from current globpri
value(decreasing the priority)/
50 pri_t ts_tqexp / ts_umdpri assigned when
proc at this level /
51 / exceeds its time quantum /
52 pri_t ts_slpret / ts_umdpri assigned when
proc at this level /
53 / returns to user mode after sleeping /
/Control I/O bound decay/
/threshhold/
54 short ts_maxwait / bumped to ts_lwait if
more than ts_maxwait /
55 / secs elapse before receiving full quantum
/
56 short ts_lwait / ts_umdpri assigned if
ts_dispwait exceeds /
57 / ts_maxwait /

14
/uts/common/disp/ts_dptbl.c

77 define TSGPUP0 0 / Global priority for TS
user priority 0 /
78 define TSGPKP0 60 / Global priority for TS
kernel priority 0 /
98 tsdpent_t config_ts_dptbl
99
100 / glbpri qntm tqexp slprt mxwt lwt /
101
102 TSGPUP00, 20, 0, 50, 0, 50,
124 TSGPUP022, 12, 12, 52, 0, 52,
136 TSGPUP034, 8, 24, 53, 0, 53,
149 TSGPUP047, 4, 37, 58, 0, 58,
159 TSGPUP057, 4, 47, 58, 0, 59,
160 TSGPUP058, 4, 48, 58, 0, 59,
161 TSGPUP059, 2, 49, 59, 32000, 59
162

15
Thread priority calculation Dispatcher
Algorithm

Quantum corresponding to the class-relative
priority
Lower priority but longer quantum to favor IA
ts_cpupri is used to index into TS disptbl and
updated itself by the corresponding ts_tqexp
user mode priority calculated
ts_globpriTSGPUPts_umdpri (used as index)
t_prits_globpri or
t_prilowest SYS priority if ts_flagsTSKPRI(indi
cates working in SYS class)

16
/uts/common/sys/ts.h

64 typedef struct tsproc
65 int ts_timeleft / time remaining in procs
quantum /
/updated per sec by ts_update() and compared
with tsdpend.ts_maxwait/
66 uint_t ts_dispwait / wall clock seconds
since start /
67 / of quantum (not reset upon preemption /
71 pri_t ts_umdpri / user mode priority
within ts class /
/adjustment ts_umdprits_cpuprits_uprits_boo
st/
68 pri_t ts_cpupri / system controlled
component of ts_umdpri /
69 pri_t ts_uprilim / user priority limit /
70 pri_t ts_upri / user priority /
74 char ts_boost / interactive priority
offset /
/distinguish between IA and TS/
75 uchar_t ts_flags / flags defined below /

17
/uts/common/sys/thread.h

106 typedef struct _kthread
/set by tsdpent_t.ts_globpri/
120 pri_t t_pri / assigned thread priority /
/scheduling-class-specific structure linked to
every kthread /
129 struct thread_ops t_clfuncs / scheduling
class ops vector /
130 void t_cldata / per scheduling class
specific data /
/inherited from parent thread, initial thread
is LWP/kthread/
121 pri_t t_epri / inherited thread priority
/
/the SYS thread priority for this thread/
/Increase the TS/IA class to SYS class if
critical resource obtained/
195 uint_t t_kpri_req / kernel priority
required /
336 kthread_t

18
/uts/common/disp/ts.c

128 define TS_NEWUMDPRI(tspp) \
129 \
130 pri_t pri \
131 pri (tspp)-gtts_cpupri (tspp)-gtts_upri
(tspp)-gtts_boost \
132 if (pri gt ts_maxumdpri) \
133 (tspp)-gtts_umdpri ts_maxumdpri \
134 else if (pri lt 0) \
135 (tspp)-gtts_umdpri 0 \
136 else \
137 (tspp)-gtts_umdpri pri \
138 ASSERT((tspp)-gtts_umdpri gt 0
(tspp)-gtts_umdpri lt ts_maxumdpri) \
139

19
1659 ts_tick(kthread_t t)

1693 tspp-gtts_cpupri ts_dptbltspp-gtts_cpupri.t
s_tqexp
1694 TS_NEWUMDPRI(tspp)
1696 new_pri ts_dptbltspp-gtts_umdpri.ts_globpr
i
1706 if ((t-gtt_schedflag TS_LOAD)
1710 tspp-gtts_timeleft
1711 ts_dptbltspp-gtts_cpupri.ts_quantum

20
TS Class vs IA Class

The same dispatcher table
IA for windows (This can be observed by last
semesters project, the active windows has more
chances to run)
Share TSs thread tsproc_t data structure by flag
ts_flags

21
TS Class vs IA Class (Cont)

ts_boost for IA (10, cancel the effect of
ts_tqexp)
ts_boost for TS (0)
IA use setfrontdq() for getting scheduled ASAP
TS use
setbackdq() for maintaining a banlance in queue
depth across processors
setfrontdq() for waiting for a while

22
/uts/common/sys/ts.h

61 /
62 time-sharing class specific thread structure
63 /
64 typedef struct tsproc
65 int ts_timeleft / time remaining in procs
quantum /
66 uint_t ts_dispwait / wall clock seconds
since start /
67 / of quantum (not reset upon preemption /
68 pri_t ts_cpupri / system controlled
component of ts_umdpri /
69 pri_t ts_uprilim / user priority limit /
70 pri_t ts_upri / user priority /
71 pri_t ts_umdpri / user mode priority within
ts class /
72 pri_t ts_scpri / remembered priority, for
schedctl /
73 char ts_nice / nice value for compatibility
/
74 char ts_boost / interactive priority offset
/
75 uchar_t ts_flags / flags defined below /
76 kthread_t ts_tp / pointer to thread /
77 struct tsproc ts_next / link to next
tsproc on list /

23
/uts/common/disp/ts.c

1653 Check for time slice expiration. If time
slice has expired
1654 move thread to priority specified in
tsdptbl for time slice expiration
1655 and set runrun to cause preemption.
1656 /
1657
1658 static void
1659 ts_tick(kthread_t t)
1668 if ((tspp-gtts_flags TSKPRI) 0)
1669 if (--tspp-gtts_timeleft lt 0)
1670 pri_t new_pri
1671
1672 /
1673 If we're doing preemption control and
trying to
1674 avoid preempting this thread, just note
that
1675 the thread should yield soon and let it
keep
1676 running (unless it's been a while).
1677 /
1678 if (t-gtt_schedctl schedctl_get_nopreempt
(t))

24
/uts/common/disp/ts.c

1653 Check for time slice expiration. If time
slice has expired
1654 move thread to priority specified in
tsdptbl for time slice expiration
1655 and set runrun to cause preemption.
1656 /
1657
1658 static void
1659 ts_tick(kthread_t t)
1668 if ((tspp-gtts_flags TSKPRI) 0)
1669 if (--tspp-gtts_timeleft lt 0)
1670 pri_t new_pri
1671
1672 /
1673 If we're doing preemption control and
trying to
1674 avoid preempting this thread, just note
that
1675 the thread should yield soon and let it
keep
1676 running (unless it's been a while).
1677 /
1678 if (t-gtt_schedctl schedctl_get_nopreempt
(t))

25
/uts/common/disp/ts.c

1686
1687 TNF_PROBE_2(schedctl_failsafe,
1688 "schedctl TS ts_tick", / CSTYLED /,
1689 tnf_pid, pid, ttoproc(t)-gtp_pid,
1690 tnf_lwpid, lwpid, t-gtt_tid)
1691
1692 tspp-gtts_flags TSRESTORE
1693 tspp-gtts_cpupri ts_dptbltspp-gtts_cpupri
.ts_tqexp
1694 TS_NEWUMDPRI(tspp)
1695 tspp-gtts_dispwait 0
1696 new_pri ts_dptbltspp-gtts_umdpri.ts_glob
pri
1697 ASSERT(new_pri gt 0 new_pri lt
ts_maxglobpri)
1698 /
1699 When the priority of a thread is
changed,
1700 it may be necessary to adjust its
position
1701 on a sleep queue or dispatch queue.
1702 The function thread_change_pri
accomplishes
1703 this.
1704 /

26
/uts/common/disp/ts.c

1698 /
1699 When the priority of a thread is
changed,
1700 it may be necessary to adjust its
position
1701 on a sleep queue or dispatch queue.
1702 The function thread_change_pri
accomplishes
1703 this.
1704 /
1705 if (thread_change_pri(t, new_pri, 0))
1706 if ((t-gtt_schedflag TS_LOAD)
1707 (lwp t-gtt_lwp)
1708 lwp-gtlwp_state LWP_USER)
1709 t-gtt_schedflag TS_DONT_SWAP
1710 tspp-gtts_timeleft
1711 ts_dptbltspp-gtts_cpupri.ts_quantum
1712 else
1713 tspp-gtts_flags TSBACKQ
1714 cpu_surrender(t)
1715
1716 TRACE_2(TR_FAC_DISP, TR_TICK,

27
Priority Inheritance

Prevent priority inversion
Each thread has two priorities global priority
and inherited priority. The inherited priority is
normally zero unless the thread is sitting on a
resource that is required by a higher priority
thread.
When a thread blocks on a resource, it attempts
to "will" or pass on its priority to all threads
that are directly or indirectly blocking it. The
pi_willto() function checks each thread that is
blocking the resource or that is blocking a
thread in the syncronization chain. When it sees
threads that are a lower priority, those threads
inherit the priority of the blocked thread. It
stops traversing the syncronization chain when it
hits an object that is not blocked or is higher
priority than the willing thread.
If someone maliciously grab some resource?

28
Mechanism / Frame

Dispatcher manage queues of runable threads
---run the highest priority thread
---recalculate the thread priority
Multi dispatch queues
one for each processor
one kernel preempt queue for systemwide
for unbound RT threads
one kernel preempt queue for each processor set
for RT threads
Double linked list
Clock-driven

29
/src/uts/common/sys/disp.h

47 typedef struct dispq
48 kthread_t dq_first / first thread on queue
or NULL /
49 kthread_t dq_last / last thread on queue
or NULL /
50 int dq_sruncnt / number of loaded, runnable
/
51 / threads on queue /
52 dispq_t

30
/uts/common/sys/disp.h

54 /
55 Dispatch queue structure.
56 /
57 typedef struct _disp
58 disp_lock_t disp_lock / protects
dispatching fields /
59 pri_t disp_npri / of priority levels in
queue /
60 dispq_t disp_q / the dispatch queue /
61 dispq_t disp_q_limit / ptr past end of
dispatch queue /
62 ulong_t disp_qactmap / bitmap of active
dispatch queues /
63
64 /
65 Priorities
66 disp_maxrunpri is the maximum run priority
of runnable threads
67 on this queue. It is -1 if nothing is
runnable.
68
69 disp_max_unbound_pri is the maximum run
priority of threads on
70 this dispatch queue but runnable by any
CPU. This may be left
71 artificially high, then corrected when some
CPU tries to take
72 an unbound thread. It is -1 if nothing is
runnable.

31
Dispatcher

ts_tick() recalculate the priority of the running
process
ts_update() recalculate the priority of the
process in a dispatch queue or sleep queue
setfrontdq()setbackdq() will cause preempt()
ts_tick() will cause cpu_surrend()
ts_tick() or ts_yield will cause swtch()
swtch() will call disp()
disp() looks for the highest-priority to run
First search kernel preempt queue
Then search the queue of the current CPU
Search the dispatch queue of other CPUs
Idle thread

32
/uts/common/disp/disp.c

685 while ((pri kpq-gtdisp_maxrunpri) gt 0
686 pri gt dp-gtdisp_maxrunpri
687 (cpup-gtcpu_flags CPU_OFFLINE) 0
/fectch the best-priority thread from the
kernel preempt queue/
688 (tp disp_getbest(kpq)) ! NULL)
695
698 pri dp-gtdisp_maxrunpri
707 if (pri -1)
708 if (!(cpup-gtcpu_flags CPU_OFFLINE))
/find a processor with the highest-priority
thread/
710 if ((tp disp_getwork(cpup)) NULL)
711 tp cpup-gtcpu_idle_thread
718
719 else
721 tp cpup-gtcpu_idle_thread
727

33
Scheduler activation

Preemption control
The management of the LWP-to-User-thread problem
The management fo keeping the correct number of
LWPs available for a threaded process

34
LWP

Lightweight process
Execute a function call to a function that is
part of another processs address space, pass the
function arguments, and get a return value as if
the function was part of the calling process
A lightweight process can be considered as the
swappable portion of a kernel thread
a lightweight process is to think of them as
"virtual CPUs" which perform the processing for
applications. Application threads are attached to
available lightweight processes, which are
attached to a a kernel thread, which is scheduled
on the system's CPU dispatch queue.
Communication between the kernel and user-level
threads library
Based on shared memory pages
System call lwp_schedctl()
primordial thread t0
sc_init() establish the shared memory pages and
the upcall door.

35
User thread vs Kernel thread

A kernel thread is the entity that is scheduled
by the kernel. If no lightweight process is
attached, it is also known as a system thread. It
uses kernel text and global data, but has its own
kernel stack, as well as a data structure to hold
scheduling and syncronization information.
Kernel threads can be independently scheduled on
CPUs. Context switching between kernel threads is
very fast because memory mappings do not have to
be flushed.
User threads are scheduled via a scheduler in
libthread. This scheduler does implement
priorities, but does not implement time slicing.
If time slicing is desired, it must be programmed
in.

36
User thread vs Kernel thread(Cont)

User thread use thread librarys own scheduler
Solaris currently ships with two threads
libraries
libthread.so, for support of the Solaris threads
interfaces, user threads are created by a call to
thr_create(3THR) (Solaris threads)
libpthread.so, the POSIX (Portable Operating
System Interface for Unix) threads APIs, user
threads are created by a call to
pthread_create(3THR) (POSIX threads).

37
User thread vs Kernel thread

The Multithreaded Process Model (1)

38
Scheduler activation

Schedctl_init() initialize, turning on premption
control.
sc_init() estalbish a door for kernel-to-user
upcalls
schedctl_block() determin if this LWP is the last
one in the process when LWP is to sleep.

39
Preemption control

In ts-tick, if ts_timeleft reaches 0, give the
kthread a few extra ticks beyond its time quantum
to free the critical resources
Get one more time slice to run, not allow the
scheduler activation to keep thread running
indefinitely.

40
Citation

1. Solaris Internals
2. http//www.princeton.edu/psg/unix/Solaris/trou
bleshoot/process.html

41
Questions ?
42
Scheduler activation

Give the kthread a few extra clock ticks beyond
its time quantum to complete its task and free
the lock
Activated by mutex lock
Only given one extra ts_tick()

43
Processor Set

RT in the same processor set
Interrupt disabled in the RT set

44
Kernel Service

System Calls The kernel executes requests
submitted by processes via system calls. The
system call interface invokes a special trap
instruction.
Hardware Exceptions The kernel notifies a
process that attempts several illegal activities
such as dividing by zero or overflowing the user
stack.
Hardware Interrupts Devices use interrupts to
notify the kernel of status changes (such as I/O
completions).
Resource Management The kernel manages resources
via special processes such as the pagedaemon.

45
Hash table