Python lessons

[Bu🤠ns]

Thanks, Tel and Trip!! I think I'm a lot better at understanding it today than I was yesterday.  I think what I need to do now is go back and play around with classes and their special methods.  I appreciate the info.

[Triple Zero]

Another thing classes in Python are useful for, is again something that's quite different from your typical OOP language such as Java:

Have you learned about decorators yet?

A decorator is simply a function that takes a function as a parameter and returns another function (usually a transformed version of the parameter). Confusing? I hope not!

It's very simple, for instance, here we declare a function that takes a parameter func (which as you can see from its name is assumed to be a callable object, a function). Then it returns a new function that simply calls func twice:

>>> def twice(func):
...     def new_func(*a, **kw):
...         func(*a, **kw)
...         func(*a, **kw)
...     return new_func

There might be two weird bits about the above code. First: yes indeed we did just define a function inside another function and that's perfectly fine, don't look at me like that. Second, if the *a,**kw bit scares you, read the docs again, but basically *a gets a list of any unnamed arguments and **kw gets a dict of any keyword arguments. It's done so it'll work with any sort of function, regardless what parameters it takes.

Let's see it in action!

>>> @twice
... def bloop():
...     print "BLLOOOP!"
...     
>>> bloop()
BLLOOOP!
BLLOOOP!

Works perfectly! You put a decorator above a function definition, with a @ in front of it, and then it'll "decorate" that function. So, by decorating the function bloop with the @twice decorator, bloop is fed to the twice function and replaced by its return value, being a function that calls bloop twice.

So how does it know to call the old bloop twice and not the newly decorated bloop? (which would result in an infinite loop of bloop, or infinite bloop, perhaps).

Well, that's another cool thing you get with defining functions inside of other functions: closures

When Python sees the definition of bloop with the decorator above it, it calls the function twice with bloop as a parameter. Inside this function, the parameter looks like a local variable named func right?

And then inside the function, we define another function named new_func. However, this new_func code contains a reference to the variable func (two, even). BUT!! The variable func is not local to new_func, it's from one level higher, it's local to twice. And we all know what happens to local variables when you exit their scope, right? They vanish! We can't have that because after we exited twice, we still got new_func which might still need to use that variable!

Fortunately, Python is very smart, recognizes that there is a local variable, used inside an inner function, and it creates a "closure".

Basically what that means is that it creates a frozen copy of the local variables referenced by new_func and attaches them to new_func to keep. And that frozen copy is our old bloop, and not the new one.

See a closure is kind of a bit like that science fiction bullshit when they say you can cut off a little bubble of space-time and create a tiny parallel universe, except without the hand-wavy stuff because Python always lets you look at its dirty insides:

>>> bloop()
BLLOOOP!
BLLOOOP!
>>> bloop.func_closure
(<cell at 0x000000000844DAC8: function object at 0x00000000082E9438>,)
>>> bloop.func_closure[0]
<cell at 0x000000000844DAC8: function object at 0x00000000082E9438>
>>> bloop.func_closure[0].cell_contents
<function bloop at 0x00000000082E9438>

There! You see it? There it is!!HA! That's our old bloop! Check it, we can call it and:

>>> bloop.func_closure[0].cell_contents()
BLLOOOP!

Just once! Whooo

(honest I never actually tried this before, but shit just works)



Next post will give another way of making a decorator!

[Triple Zero]

There is another way besides making a function in a function! Remember how a class works?

To make an object instance from a class, you basically "call" it, right?

Actually that's exactly what happens, not just "basically". SO if the __init__ method of the class accepts a function as the second parameter (because the first one is always the implicit "self") ... then ... are you feeling it?

... then the __init__ function returns the new instance and the decorated function is replaced with that instance object.

so ... when people might try to call this decorated function, that means the object must know what to do, for which we can use the special method __call__ !!

>>> class twice(object):
...     def __init__(self, func):
...         self.func = func
...     def __call__(self, *a, **kw):
...         self.func(*a, **kw)
...         self.func(*a, **kw)

See? It can be used exactly the same as the previous decorator:

>>> @twice
... def bloop():
...     print "BLLOOOP!"
...     
>>> bloop()
BLLOOOP!
BLLOOOP!

Except that it doesn't use a closure, but it saves the old version inside the object instance itself, as a property called "func":

>>> bloop
<__main__.twice object at 0x000000000844EB70>
>>> bloop.func
<function bloop at 0x0000000008EC80B8>
>>> bloop.func()
BLLOOOP!


Next post: doing something USEFUL with a decorator!

[Triple Zero]

Ok say you made a very cool class for playing Rock Paper Scissors, like this one:

>>> class Roshambo(object):
...     options = ['ROCK','PAPER','SCISSORS']
...     def __init__(self, what=None):
...         if what not in Roshambo.options: # if we don't supply a choice, pick a random one
...             what = random.choice(Roshambo.options)
...         self.what = what
...     def __str__(self):
...         return self.what # just useful for printing
...     def __eq__(self, other):
...         return self.what == other.what
...     def __gt__(self, other):
...         if self.what == 'ROCK' and other.what == 'SCISSORS':
...             return True
...         if self.what == 'PAPER' and other.what == 'ROCK':
...             return True
...         if self.what == 'SCISSORS' and other.what == 'PAPER':
...             return True
...         return False

See it's got a special method __str__ so you can print it and see what's inside. And it also got a method __eq__ (equals) so you can compare two players and see if they tie, and finally __gt__ (greater than) so you can see if one player wins over the other. Like this:

>>> trip = Roshambo()
>>> burns = Roshambo()
>>> if trip > burns:
...     print "Trip wins!"
... elif trip == burns:
...     print "It's a tie!"
... else:
...     print "Burns wins!"
...     
It's a tie!
>>> print trip, burns
PAPER PAPER

Seems to work, right?

But what if you'd also want to use the < operator? And maybe even the <= and the >= and the != operators? Well you could write a special method for all of those, but that would get really boring. And besides, we already defined == and >, so from there Python could figure out how the other operators work, right? Because A<B means the same as "not A>B and not A==B" and so it goes.

Turns out, Python is smart like that! And you can import a special decorator from the functools module.

Oh btw did I mention that you can decorate classes in exactly the same way as you can do functions? Well, you can.

like this:

>>> import functools
>>> @functools.total_ordering
... class Roshambo(object):
...     options = ['ROCK','PAPER','SCISSORS']
... # ... etc rest of the class is identical
... # ...

>>> trip = Roshambo()
>>> burns = Roshambo()
>>> if burns < trip:
...     print "Trip wins!"
... elif trip == burns:
...     print "It's a tie!"
... else:
...     print "Burns wins!"
...
Trip wins!
>>> trip >= burns
True

See? I WON!!!

I mean, also what's nice is that we can use all of the comparison operators, by just defining two of them, and the @functools.total_ordering decorator will figure out the rest!

Well. And that is nice.

[Bu🤠ns]

Thanks, Trip...

So now I'm trying to code something that will create an .m3u playlist file from the contents of a directory.

What I want it to do is to request two user inputs:

1. It requests a folder path to the folder containing mp3s (to be later updated to include a 'browse gui' once I'm ready to open up the gui can of worms)
2. I want it to request a playlist name, or, if just left blank to grab the folder basename and concatenate that to '.m3u'.

Now I have a working script that will do exactly that:

Code Select Expand


import os
import glob

directory = raw_input("Enter file folder path: ")
os.chdir(directory)
playlist_title = raw_input("Enter playlist title (optional): ")

if not playlist_title:
    newtitle=os.path.basename(directory)
    for files in glob.glob("*.mp3"):
        f = open(newtitle+".m3u", "a")
        f.writelines(files+"\n")
        f.close()
else:
    for files in glob.glob("*.mp3"):
        f = open(playlist_title+".m3u", "a")
        f.writelines(files+"\n")
        f.close()


But since I'm trying to understand OOP and classes I've been trying to re-write it in that form.
So far this is what I have:

Code Select Expand



class playlist:
    def change_directory(self, filepathlocation):
        '''moves script to working dir'''
        os.chdir(filepathlocation.replace('"', ''))

    def write_the_file(self,title):
        '''writes the .m3u file'''
        for files in glob.glob("*.mp3"):
            f = open(title + ".m3u", "a")
            f.writelines(files+"\n")
            f.close()

    def playlist_title(self, title):
        '''creates the m3u playlist title either from user input or directory basename'''
        if title == None:
            title = os.path.basename(playlist.change_directory(filepathlocation))
            playlist.write_the_file(title)
        else:
            playlist.write_the_file(title)

playlist = playlist()
filepathlocation = playlist.change_directory(raw_input("Enter file folder path "))
title = playlist.playlist_title(raw_input("Enter playlist title (optional): "))


Now, where I'm stuck is in the playlist_title() function.  When I just leave the playlist title empty it does complete but it doesn't concatenate the input variable 'title' to the '.m3u' extension.  IOW, I get an .m3u file titled '.m3u'. 

Questions:
1. In flow control, what is the difference between if title == None: and IF NOT title: ?  Both appear to be the same thing and seem to work the same depending on how I write the code.  Is it better, or perhaps more 'pythonic' to write one over the other?

2. Why does os.path.basename(directory) work in my original, process oriented script but not in my object oriented script?

Thanks, folks

ETA: I realize that the original form of the script is probably a better way to accomplish this task (especially in regard to the video earlier ITT), but since I'm still trying to wrap my head around classes and oop, I figured the script was easy enough to duplicate.

[Triple Zero]

First, you probably don't want to read from standard input with raw_input, instead you want to make a commandline tool and use the argparse module (see python docs).

The rest of your code I'll have a look at later.

About making GUI things, I don't have a lot of experience with that but there's probably some good tutorials out there.

QT is a good GUI library I've heard good things about. It also has Python-bindings, afaik. These articles use C++, but might still be some help (or not) : http://www.tuxradar.com/learnqt

[Bu🤠ns]

i got it... Enki-][ helped a bit in IRC.

The current working directory parameter was outside the scope of the playlist_title function. (I think I said that correctly).

That's interesting how that all works out. I can see I definitely need to become more aware of this.

Code Select Expand


import os,glob

class playlist:
    def change_directory(self, filepathlocation):
        '''moves script to working dir'''
        os.chdir(filepathlocation.replace('"', ''))

    def write_the_file(self,title):
        '''writes the .m3u file'''
        for files in glob.glob("*.mp3"):
            f = open(title + ".m3u", "a")
            f.writelines(files+"\n")
            f.close()

    def playlist_title(self, title):
        '''creates the m3u playlist title either from user input or directory basename'''
        if not title:
            cwd = os.getcwd()
            title = os.path.basename(cwd)
            playlist.write_the_file(title)
        else:
            playlist.write_the_file(title)

playlist = playlist()
filepathlocation = playlist.change_directory(raw_input("Enter file folder path "))
title = playlist.playlist_title(raw_input("Enter playlist title (optional): "))


[Triple Zero]

I wonder if this might be useful to any of you guys:

www.ironspread.com -- Script Excel with Python (and here's the HN THread)

[Bu🤠ns]

TRIP! THANKS!  I was looking for something like this!! I kept putting off learning more VBA because, well frankly it's so clunky.  I have some comma separated values and other data I've been needing to organize with Excel but didn't have the time to go diving into VBA. 

[Triple Zero]

[Golden Applesauce]

Questions:
1. In flow control, what is the difference between if title == None: and IF NOT title: ?  Both appear to be the same thing and seem to work the same depending on how I write the code.  Is it better, or perhaps more 'pythonic' to write one over the other?

"title == None" checks for exactly what you'd expect - that the value of title is None. "if not title" will check if title is any 'falsey' value, of which there are many: None, False, 0, "", [], and maybe some others. (I think you can give your own classes truthiness values?)

In general, you probably want the explicit test whenever possible - it's easy to make code errors by accidentally testing for something that you didn't mean to. For example, you might have a TryParseToInt(string) function that converts a string to an integer, or None if the string didn't represent an integer (to avoid throwing an exception.) Code like:

num = TryParseToInt(user_input)
if num:
  # valid input case - do stuff
else:
  # re-ask for input

will not do what you expect in the event the user enters "0" - which is a value that your code wants to treat as valid but the if statement treats as false.

[Bu🤠ns]

Questions:
1. In flow control, what is the difference between if title == None: and IF NOT title: ?  Both appear to be the same thing and seem to work the same depending on how I write the code.  Is it better, or perhaps more 'pythonic' to write one over the other?

"title == None" checks for exactly what you'd expect - that the value of title is None. "if not title" will check if title is any 'falsey' value, of which there are many: None, False, 0, "", [], and maybe some others. (I think you can give your own classes truthiness values?)

In general, you probably want the explicit test whenever possible - it's easy to make code errors by accidentally testing for something that you didn't mean to. For example, you might have a TryParseToInt(string) function that converts a string to an integer, or None if the string didn't represent an integer (to avoid throwing an exception.) Code like:

num = TryParseToInt(user_input)
if num:
  # valid input case - do stuff
else:
  # re-ask for input

will not do what you expect in the event the user enters "0" - which is a value that your code wants to treat as valid but the if statement treats as false.

Thanks for that detail  

It makes sense that if the code needs to be specific then it should be specific.

[Golden Applesauce]

Thanks for that detail  

It makes sense that if the code needs to be specific then it should be specific.

Looking back at your code, this was actually one of your problems in the non-working version.

Code Select Expand



class playlist:
    def change_directory(self, filepathlocation):
        '''moves script to working dir'''
        os.chdir(filepathlocation.replace('"', ''))

    def write_the_file(self,title):
        '''writes the .m3u file'''
        for files in glob.glob("*.mp3"):
            f = open(title + ".m3u", "a")
            f.writelines(files+"\n")
            f.close()

    def playlist_title(self, title):
        '''creates the m3u playlist title either from user input or directory basename'''
        if title == None: # not what you meant!
            title = os.path.basename(playlist.change_directory(filepathlocation))
            playlist.write_the_file(title)
        else:
            playlist.write_the_file(title)

playlist = playlist()
filepathlocation = playlist.change_directory(raw_input("Enter file folder path "))
title = playlist.playlist_title(raw_input("Enter playlist title (optional): "))


Now, where I'm stuck is in the playlist_title() function.  When I just leave the playlist title empty it does complete but it doesn't concatenate the input variable 'title' to the '.m3u' extension.  IOW, I get an .m3u file titled '.m3u'. 

raw_input() always returns a string of some kind - if the user doesn't actually type any characters, it just returns the empty string "".  Since "" != None, you were always hitting the else: case and calling write_the_file() with an empty string. So your code never actually tried to evaluate filepathlocation - if it had, it would have bugged out and given you "NameError: name 'filepathlocation' is not defined" instead of completing correctly. (Since it wasn't in scope, as you learned from Enki.) And if filepathlocation were in scope, you would have run into another problem which is that change_directory doesn't return a value returns None (as it should), so you'd have been calling os.path.basename() on None, which throws a exception. (Note that in your current program, filepathlocation, which is assigned to but never used, has a value of None after the program executes.)

There's a really subtle bug in your current working program - try changing the variable named "playlist" to something else, like "the_playlist".

Code Select Expand


playlist = playlist()
filepathlocation = playlist.change_directory(raw_input("Enter file folder path "))
title = playlist.playlist_title(raw_input("Enter playlist title (optional): "))

to

Code Select Expand


the_playlist = playlist()
filepathlocation = the_playlist.change_directory(raw_input("Enter file folder path "))
title = the_playlist.playlist_title(raw_input("Enter playlist title (optional): "))


The resulting error, the fix, and how your code managed to work anyway, will blow your mind.

[Golden Applesauce]

On object-oriented programming in general -
It's not easy to "get" OOP until you've had to maintain bad OOP code and had the pleasure of working on good OOP code. The defining trait of bad code isn't that it fails to work correctly, but that modifying it safely is a pain in the ass that forces future developers working on your project to make compromises. So code that is only really going to be run a handful of times and only ever in one environment - like your playlist making script - never needs to be updated, so putting in the effort to do OOP correctly feels like wasted effort because you're not around to see the benefits.

If you're interested in really learning how to do object orient programming correctly, I'd recommend looking into SOLID programming, the DRY principle, and related terms. Uncle Bob's Clean Code is also a great book.

With all that said - your current script really doesn't take any advantage of being object oriented. The whole program would be vastly simplified if you dropped the class declaration and moved all the method declarations to global functions; if nothing else, that saves you from having to instantiate a playlist() object. That we can do that is our first code smell - playlist() encapsulates no state. This isn't necessarily a bad thing, but in our case it is, because the program is keeping track of some state: the current working directory.  It's just doing so by setting the CWD for the entire program, which is wrong because only playlist cares about the CWD, so only the playlist should know about / have the opportunity to muck with the CWD. If some other component of the program also cared about the CWD, it would have to dance around playlist to avoid messing it up, which is a lot of extra effort and opportunities to create bugs.

So the first refactoring I'd do would be to change playlist.change_directory to something like this, with the corresponding change in write_the_file:

Code Select Expand


def change_directory(self, current_directory):
  self.current_directory = current_directory

...

def write_the_file(self, title):
  ...
  f  = open(self.current_directory + title + ".m3u", "a")
  ...


Note that I left off the logic that deals with stripping the quotes off of the user input. That's related to the Single Responsibility Principle, which basically says that each class should do one and only one thing. Worrying about how to parse user input is someone else's job - we'll just say that it's the responsibility of whoever calls change_directory make sure that they provide a valid current_directory parameter.

That brings us to another responsibility that playlist has taken upon itself, which is that it finds all of the .mp3 files in addition to creating a playlist based on them.  This means that if we wanted to change the way we get music filenames (maybe you reorganize your collection and now want to get relative paths to individual .mp3s inside a music/artist/album/song.mp3 folder structure, or you get some .ogg's, or you want do the same thing for videos, or you want the songs sorted in a particular way) we have to open up playlist's source and start tweaking things. This is where OOP starts to shine - we can create a separate object that's responsible for getting which files we want, and then have it tell playlist what files to include in the playlist. If we have more than one way in which we want to get the filenames, we just make new implementations of GetFilenames or whatever and use them with playlist - currently, we have to make an entirely new playlist class and duplicate all the other playlist logic if we want to get different files.

There are two basic ways to go about this.  The first is to simply extract glob.glob("*.mp3") to a parameter, so you end up with something like:

Code Select Expand


def write_the_file(self, title, file_names):
  f = open(self.current_directory + title + ".m3u", "a")
  for file_name in file_names:
    f.write(file_name) # let file.write worry about line endings for you
  f.close()


where you just run glob.glob when you call write_the_files. The advantage to this is that if we wanted to add multiple groups of files to a single playlist, we can just call write_the_file with a different list of file_names and they'll all get appended to the same playlist file.  The disadvantage of this is that there's a good chance our file-finding function is going to depend on CWD (like glob.glob does), which would require us to duplicate code - we have to tell playlist and our file-finding source what CWD is separately. We can fix that by passing either a function or an object exposing a specifically-named method to write_files instead of just the list:

Code Select Expand


def write_the_file(self, title, GetFilesFromCWD)
  ...
  for file_name in GetFilesFromCWD(self.current_directory):
  ...

the_playlist.write_the_file("cool tunes", lambda cwd: glob.glob(cwd + "*.mp3"))


The only potential problem here is that if we really do want to get our music files in the same way most of the time, we end up repeating our file-getter expression every time we need to call write_the_file.  That's not very DRY, so in that use case I'd recommend using composition: each playlist object will simply contain another object that knows how to get a list of files to be added to the playlist.

Code Select Expand


class playlist:
  def __init__(self, file_name_getter):
    self.file_name_getter = file_name_getter
  ...
  def write_the_file(self, title):
  ...
    for file_name in self.file_name_getter.get_files(self.current_directory)
    ...


This is perfect for the case where the main use of a playlist object is going to be to create a number of different playlists, each in a different working directory and/or with a different title, but each time you want to the find music files in the same way relative to the current directory. If the main use of a playlist would be to sit in one directory and append different groups of songs to the same playlist, I'd go with the earlier approach of just passing either the list of file names or a function to generate them directly, and put title on the __init__ so I didn't have to pass the same title over and over. Then we could drop playlist_title and either move its title-fixing logic into the constructor or just require the caller to pass a valid title. In fact, in that case I'd require users of playlist to pass a valid directory into the constructor and do away with the change_directory method; if every instance of playlist needs it's directory to be set exactly once, then I don't want to allow people to forget to do it and there's no need to go to extra effort to allow people to set it more than once. That would be a more subtle case of violating DRY - requiring both foo = playlist() and foo.change_directory() is like making someone order their water and a glass to hold it separately; the former implies the latter.

[Bu🤠ns]

http://pyvideo.org/  ooooooooo