Binary Compatibility of C shared libraries on GNU/Linux.

1
Binary Compatibility of C shared libraries on
GNU/Linux.
SYRCoSE Spring Young ResearchersColloquium on
Software EngineeringMay 28-29, 2009, Higher
School of EconomicsMoscow, Russian Federation

• Pavel Shved
• Denis Silakov
• Institute for System Programming, RAS

2
Who cares?

• C 3rd place in TIOBE index.
• Shared libraries libstdc, Qt, KDE-libs
• GNU/Linux
• Component approach to system architecture
• Likes to follow standard
• Open-source
• A problem will to improve the library vs.
  retaining compatibility

3
Existing research

• KDE compatibility Guide
• Nokias Qt practical experience summary
• Random messages on boards and ml-s
• Truly outdated references
• Mostly in form Thou shalt not do anything.

4
Our research
Library spec-s
Set of Library changes
Set of programs
5
Aims of the research

• Formally study Cs compatibility issues
• Model for build process
• Standards
• Itanium C ABI, rev. 1.86
• ISO/IEC 148822003 Programming languages C
• GCC compiler itself
• Constrain the set of possible library changes for
  its different specifications

6
Source and binary compatibility

• p program L library v.1 L library v.2
• such that defined
• p.exe p.compile(L.h).link(L.so)

p.compile(L.h).link(L.so).dynlink(L.so) p.compile
(L.h).link(L.so).dynlink(L.so)
p.compile(L.h).link(L.so).dynlink(L.so)
Source compatibility
Binary compatibility
7
Symbol versioning

• Modified L.h listing
• namespace lib_L
• include ltL.hgt
• using namespace lib_L

• Modified L.h listing
• namespace lib_L
• include ltL.hgt
• namespace lib_L
• include ltL.hgt
• using namespace lib_L

Code duplicaton 25 minor releases of Qt 4 so
far, 50 Mb each. A nightmare.
8
Dynamic linker errors

• Declaration ? symbol(s)
• Name mangling
• Keep symbols, that can be referenced.

?
void X(int, float)
_Z8Functionif
?
void X(int, int)
_Z8Functionii
_ZN5KlassC1Ev _ZN5KlassC2Ev _ZN5KlassD0Ev _ZN5Klas
sD1Ev _ZN5KlassD2Ev _ZTI5Klass _ZTS5Klass _ZTV5Kla
ss
class Klass Klass() virtual Klass()
?
9
Declarations that emit symbols

• Non-inline non-template functions
• Non-inline explicit template instantiations
• Non-template/explicitly instantiated dynamic
  classes (vtable(s), VTT)
• Problem arises only when function is called from
  the user-space.

?
void X(int, float)
_Z8Functionif
?
inline void X(int, int)
Ø
10
Name mangling caveats
typedef float Type void X(int, Type)
?
_Z8Functionif
typedef int Type void X(int, Type)
_Z8Functionii
?
class Klass void X(Klass)
?
_Z1XP5Klass
?
void X(int) void X(int, int 0)
_Z1Xi
?
_Z1Xii
templateltclass Param, class Arggt Param
X() Klass XltKlass,Typegt()
?
_Z1XI5KlassiEPT_v
11
Implementation change

• Same results on the domain of Ls version
• Bugfixes break compatibility!

Output
Input
L
L
12
Compiled code notions

• Two ways of misunderstanding
• Program p expects Ls layout, but gets Ls
• Implementation of L expects p to supply data as
  in Ls layout, but gets Ls

Klass ptrmalloc(sizeof_Klass) Klass_Ctor_1(ptr)

?
Klass ptrnew Klass()
class MyClass public Klass
MyClass ptr malloc(sizeof_MyKlass) Klass_Ctor_
2(ptr)
?
13
What changes size/layout?

• Change of the order of non-empty classes in
  hierarchy
• Sometimes change of the order of zero-sized
  classes
• Adding a member to all but the class, whose
  member will be the last, or derived from it
• Use d_ptr and do not change hierarchy.

Primary base
Next base
Virtual base that is primary for no one
Virtual base that is primary for no one
14
Virtual tables
vcall vbase offsets

• Pointer to Run-Time Type Information (RTTI)
• Offset-to-top from subobject the sub-vtable
  corresponds to
• Function pointers
• Grows around point of origin
• Used in certain virtual function calls

RTTI offset
0
Primary bases virtual function ptrs
vcall vbase offsets
RTTI offset
Next class vfunc ptrs (with its primary bases)
Virtual bases vtable that is primary for no one
Virtual bases vtable that is primary for no one
Class ptr . . . ptr -gt virtual_func()
15
Vtable properties
vcall vbase offsets

• Emitted for all dynamic classes
• Doesnt reference other
• It is constructor who fills sub-vtable pointers
• Offset to particular function is compiled into
  caller
• In the virtual function call any sub-vtable may
  be used

RTTI offset
Primary bases virtual function ptrs
vcall vbase offsets
RTTI offset
Next class vfunc ptrs (with its primary bases)
Virtual bases vtable that is primary for no one
Virtual bases vtable that is primary for no one
16
Arbitrary vtable change
vcall vbase offsets

• Force user to derive class (this will emit a copy
  of vtable in the application)
• Do not call virtual functions through pointer to
  base class, esp. in Ls code
• (or can we? Tricky wrapper?)
• The calls via pointer to derived class will use
  the copy, not Ls
• Keep an eye on thunks
• You still cant change memory layout

RTTI offset
Primary bases virtual function ptrs
vcall vbase offsets
RTTI offset
Next class vfunc ptrs (with its primary bases)
Virtual bases vtable that is primary for no one
Virtual bases vtable that is primary for no one
17
Results

• The most complete (and, thereby the least useful)
  compatibility guide
• Restrictions upon library usage do not provide
  much relaxation
• Non-derivable classes without ctors and new()
• Classes with requirement to be derived in p.
• C ABI needs redesign (Qt actually made some
  effort) to be more compatible

18
Further research

• New ABI?
• Shim layer for virtual functions (thinner than
  Qts MOC)?
• Try to build a useful preprocessor to implement
  vtable trick?

19
Thank you-)

• Pavel Shved ltshved_at_ispras.rugt
• Denis Silakov lturagan_at_ispras.rugt