Savory Flavor of Py3'0

1
Savory Flavor of Py3.0

• PyCon.it 2008
• Raymond Hettinger

2
Two versions of Talk

• Read Whats new in 3.0
• You can do that yourself -)
• Instead, lets talk about the reasoning behind
  3.0 changes, issues that arise, and how it feels
  to use the new Python.

3
Dinner is Served

• News from yesterday
• Py2.6a3 is released
• Py3.0a5 is released
• The betas releases come out next month

4
First Tasting

• Familiar, yet foreign
• Distinct dishes, not a stew
• Old dishes have fewer ingredients
• New dishes have a lot of spice
• One or two are acquired tastes
• Must be embraced to be enjoyed

5
Preparing the Kitchen

• There is a plan for migrating code from 2.5
• Almost all code in 2.5 runs in 2.6
• Many new 3.0 features are backported to 2.6
• So, youre already half-way done!
• Run 2.5 code in 2.6
• Start using 2.6/3.0 new features
• Run python -3 to discover changes that work in
  2.6
• Run the python 2-to-3 converter
• Run unittests!

6
Appetizer Transitioning Code

• Py2.5 print 2 , 11 / 5
• Py2.6 unchanged
• -3 warning floor division 11/5
• New 2.6 print 2 , 11 // 5
• 2-to-3 print (2 , 11 // 5)
• A somewhat painless transition to new code!

7
Add Taste by Removing Ingredients

• Classic classes finally gone
• One way to do it !!!
• Backticks for repr() finally gone
• No more cmp function for list.sort()
• Many outdated library modules removed
• Ints and longs are now the same object
• Universal ordering is gone. No more comparing
  strings to numbers ?

8
The Taste of Removed Features

• seq.sort(lambda x,y cmp(x.lower(), y.lower())
• seq.sort(keystr.lower)
• Cleaner, faster, less-to-know, easily understood
• Old trick decorate-sort-undecorate
• New trick rely on sort stability
• seq.sort(keyitemgetter(department))
• seq.sort(keyitemgetter(paygrade),
  reversedTrue)

9
But Someone Likes the Old Recipe

• Some APIs allow clients to specify a cmp
  function.
• Q. What do we do now the cmp is gone?
• A. Provide a converter
• def CmpToKey(mycmp)
• 'Convert a cmp function into a key
  function'
• class K(object)
• def __init__(self, obj)
• self.obj obj
• def __lt__(self, other)
• return mycmp(self.obj, other.obj)
  -1
• return K

10
Subtle Flavor Shifts

• True division 3 / 5 ? 0.6
• Old way surprised newbies
• Old way lead to subtle bugs sum(seq)/len(seq)
• Old way required coercions
• float(x)
• (0.0 x)
• Less error-prone exception syntax
• except KeyError, k ? except KeyError as k
• Cleaner calls to advance iterators
• it.next() ? next(it)
• Class decorators
• nearly zero learning curve ?
• easier than metaclasses ?
• New format for binary and octal literals
• 0b10101 and 0o621
• No more weird errors with 0666 interpreted as
  octal ?

11
More Subtle Flavor Shifts

• xrange() is now range()
• looks and feels much cleaner ?
• map() and filter() return iterators
• better use of memory
• faster
• awkward to use at the interactive prompt ?
• also, map() no longer has a special case for None
• New version of super()
• no longer need to give it arguments ?
• huge improvement over current weird requirements

12
Print is now a Function

• Simple change
• print hello ? print(hello)
• print gtgt stderr, m ? print(hello,
  filestderr)
• At first, it feels like a whole new flavor
• After a while, it feels light and clean
• Frees-up a keyword ?
• Easily substitute your own print function ?
• Easier to substitute logging calls
• Has more options and better readability

13
Minor new dish Named Tuples

• Lets you access tuple fields by name
• t0 t4 ? t.finished t.inprocess
• Michele has a whole presentation on this
• For now, just know that it will improve your life
  quite a bit if youre a frequent user of SQL or
  CSV tables
• Shows-up throughout the language in the form of
  self-documenting return tuples
• doctest.testmod() used to return a cryptic tuple
• (0, 30)
• Now it returns
• TestResults(failed0, attempted30)
• Gives the whole language a delightful, clean
  taste ?

14
Major new dish Dict views

• dict.keys/values/items now returns a view
• the view updates when the dict changes
• only interesting if mutating while iterating
• the view also provides set operations
• set ops work for items as well as keys!

15
Commentary Dict views

• Looks a bit odd at first
• Borrowed from Java
• Saves dict-to-set conversions ?
• Increases learning curve ?
• Confuses the idea of sets with dicts ?
• Doesnt come-up much
• Nice to have when you need it (esp. item sets)
• Feels more sophisticated and better specified ?
• Community reaction still unknown

16
Major new dish Star Args

• Keyword-only arguments (a big win)
• def zip_longest(args, fillvalueNone)
• Unpacking variable length inputs (a nice win)
• first, second, rest iterator
• Fully generalized unpacking (feels weird)
• first, middlevalues, last iterator

17
Major new dish Bytes and Text

• In Py2.x, we have Str and Unicode
• In Py3.0, we have Bytes and Text
• Text abstracted from encoding
• Bytes just an array of 8-bit values
• The change is long overdue other languages
  already handle this well

18
Py3.0 Text Basics

• The default encoding in Py3.0 is UTF-8 ?
• Eliminates -- coding iso-8859-1 --
• Say goodbye to ordinal not in range(128) ?
• Now its easy to follow the golden rules
• Decode early, use Unicode everywhere, encode late
• No more utext syntax ?
• file.open() can take a coding argument like
  codecs.open()
• Will guess the encoding if not specified
• Except for UTF-8, guessing difficult to do
  reliably

19
Major new flavor format() function

• Welcome addition
• Formats a single value
• Works with well-known formatting codes
• gtgtgt format(10.0, "7.3g")
• ' 10
• Theres a new, happy, wonderful addition
• User-defined formatting
• The entire transformation is fully programmable ?

20
A flavor that spreads Anchovies

• Customizable formats let us unify all the
  disconnected little formatting languages that
  pervade the language
• For instance, datetime not longer needs its own
  unique formatting engine
• Instead, it defines custom codes
• "Today is 0a b d HMS Y".format(datetime.now(
  ))
• One ring to bind them all ?

21
Commentary string formatting

• Adds to the learning curve
• many options
• complex API for custom formatters
• Unifies the language with a single formatting
  engine
• Borrowed for proven successes in C
• Keep the old formatting around for a while
• Must be embraced. Definitely, an acquired taste

22
A entirely different cuisineAbstract Base
Classes

• Inspired by DictMixin
• Inspired by Java
• Inspired by Zope Interfaces
• Good news ABCs finally define formerly vague
  notions like What is a mapping or sequence?

23
Example ABC Sequences

• class Sequence(Sized, Iterable, Container)
• """All the operations on a read-only
  sequence.
• Concrete subclasses must override __new__ or
  __init__,
• __getitem__, and __len__."""
• _at_abstractmethod
• def __getitem__(self, index)
• def __iter__(self)
• def __contains__(self, value)
• def __reversed__(self)
• def index(self, value)
• def count(self, value)

24
ABC Mix-in Example Set

• class ListBasedSet(collections.Set)
• ''' Alternate set implementation favoring
  space over speed
• and not requiring the set elements to be
  hashable. '''
• def __init__(self, iterable)
• self.elements lst
• for value in iterable
• if value not in lst
• lst.append(value)
• def __iter__(self)
• return iter(self.elements)
• def __contains__(self, value)
• return value in self.elements
• def __len__(self)
• return len(self.elements)
• s1 ListBasedSet('abcdef')
• s2 ListBasedSet('defghi')
• overlap s1 s2 The __and__()
  method is supported automatically

25
ABCs as Types

• An ABC can be a base class or can be registered
• Registration works by overriding isinstance()
  instance(3, numbers.Integral)
• New syntax for specifying the type of function
  arguments
• def f(x Integral, yRational, sSequence)
• . . .
• Not yet developed tool for validating that
  registered classes supply all the abstract
  methods

26
ABCs for Generic Functions

• To dispatch code based on argument type, ABCs are
  better than checking for indicator methods
• def f_oldway(arg)
• if hasattr(arg, __getitem) handle_map(arg)
• if hasattr(arg, write) handle_file(arg)
• ...
• def f_newway(arg)
• if isinstance(arg, Mapping) handle_map(arg)
• if isinstance(arg, FileIO) handle_file(arg)
• ...

27
Commentary Abstract Base Classes

• Mix-in behavior is very nice
• Validation of interface compliance is nice
• Clear definition of Mapping and Sequence is
  welcome
• May signal the arrival of interface checking
• Hopefully, does not give rise to tons of
  isinstance() checks
• Time will tell whether ABCs help or hurt duck
  typing

28
Other Goodies

• Buffer API with memoryview() interface
• JSON module
• New itertools
• product() cartesian product
• combinations()
• permutations()
• zip_longest()

29
Overall Thoughts

• Py3.0 is cleaner and lighter
• There are fewer pitfalls
• At first, it will feel foreign
• It has not yet been optimized
• Real world experiences are needed before we know
  whether some features will be successful
• It still looks like Python and its still fun ?

30
Question and Answer Period

• Can help explain why some changes were made
• Can discuss how each change feels when writing
  programs
• Can discuss porting issues